Table 2 - uploaded by Caroline G L Cao
Content may be subject to copyright.
Sample CUSUM chart calculation for representative subject (medical student no. 8) using VBLaST-PT Ó intermediate criterion score of 64
Source publication
Background:
Mastering laparoscopic surgical skills requires considerable time and effort. The Virtual Basic Laparoscopic Skill Trainer (VBLaST-PT(©)) is being developed as a computerized version of the peg transfer task of the Fundamentals of Laparoscopic Surgery (FLS) system using virtual reality technology. We assessed the learning curve of trai...
Similar publications
Background:
Mastery of laparoscopic skills is essential in surgical practice and requires considerable time and effort to achieve. The Virtual Basic Laparoscopic Skill Trainer (VBLaST-PC(©)) is a virtual simulator that was developed as a computerized version of the pattern cutting (PC) task in the Fundamentals of Laparoscopic Surgery (FLS) system....
Training improves skills in minimally invasive surgery. This study aimed to investigate the learning curves of complex motion parameters for both hands during a standardized training course using a novel measurement tool. An additional focus was placed on the parameters representing surgical safety and precision. Fifty-six laparoscopic novices part...
Citations
... Other common uses of VR in health professions education have been for training of technical competencies, particularly in surgical simulations for medical education and psychomotor skill acquisition (Jiang et al., 2022). In recent years, there has been a surge of research investigating the efficacy of different VR applications developed to assist medical trainees in honing laparoscopic and orthopedic surgical skills (Banaszek et al., 2017;Brinkmann et al., 2017;Rahm et al., 2018;Zhang et al., 2013). Huang & Liaw (2018) and Sultan et al. (2019) noted that the use of VR to teach and practice clinical skills to decrease the risks of causing "harm" to patients, other students, or standardized patient actors is increasing. ...
Teaching competencies for psychomotor skill development for manual handling techniques in the cervical regions is necessary for safe practice in physiotherapy. However, grasping anatomy and palpation can be challenging for students, and practice in the lab can lead to discomfort for students. To facilitate teaching and learning of this complex skill, we worked in partnership with a virtual reality (VR) industry partner who developed a customized VR application focusing on transverse ligament stress testing for manual therapy skills for Master of Physical Therapy (MPT) students. In this scholarship of teaching and learning (SoTL) project, eight MPT students participated in the evaluation of an innovative VR learning experience for manual therapy in the cervical spine. Students’ learning experiences with the custom virtual reality learning application were assessed using an observational study design with semi-structured interviews. Interview questions aligned with constructs that are recommended to assess learners’ attitudes toward VR environments. Student participants appreciated the usefulness of the application for studying and practicing the transverse ligament stress test and provided recommendations for enhancing the learning experience.
... According to our data, there are sixteen research clusters with slightly different research directions from an international perspective. For example, the researches of Aggarwal R et al.'s collaborative team (pink cluster in Fig. 2C) mainly involved the course construction of XR trainer-based ophthalmology training [37], colonoscopy training [24], laparoscopic cholecystectomy [38], endovascular techniques [39]; the work of Konge et al.'s (ranked 1st in total cooperation intensity) collaborative team (green cluster in Fig. 2C) was focused on the XR-based thoracoscopic surgery [40]; Ahmed et al.'s (ranked 2nd in total cooperation intensity) collaborative team (light blue cluster in Fig. 2C) focuses on XR-based training for robotic surgery [41], endovascular surgery [42], and endoscopic surgery [43]; the collaborative team of De et al. (ranked fifth in total collaboration intensity) (purple cluster in Fig. 2C) focuses on the construction and validation of an XR-based laparoscopic simulator [44], electrosurgery skill simulator [45], endoscopic surgical trainer [46] and suturing simulator [47]. In the case of China, there are now eight clusters of closely related scholars. ...
Objectives
The prospect of extended reality (XR) being integrated with surgical training curriculum has attracted scholars. However, there is a lack of bibliometric analysis to help them better understand this field. Our aim is to analyze relevant literature focusing on development trajectory and research directions since the 21st century to provide valuable insights.
Methods
Papers were retrieved from the Web of Science Core Collection. Microsoft Excel, VOSviewer, and CiteSpace were used for bibliometric analysis.
Results
Of the 3337 papers published worldwide, China contributed 204, ranking fifth. The world's enthusiasm for this field has been growing since 2000, whereas China has been gradually entering since 2001. Although China had a late start, its growth has accelerated since around 2016 due to the reform of the medical postgraduate education system and the rapid development of Chinese information technology, despite no research explosive period has been yet noted. International institutions, notably the University of Toronto, worked closely with others, while Chinese institutions lacked of international and domestic cooperation. Sixteen stable cooperation clusters of international scholars were formed, while the collaboration between Chinese scholars was not yet stable. XR has been primarily applied in orthopedic surgery, cataract surgery, laparoscopic training and intraoperative use in neurosurgery worldwide.
Conclusions
There is strong enthusiasm and cooperation in the international research on the XR-based surgical training. Chinese scholars are making steady progress and have great potential in this area. There has not been noted an explosive research phase yet in the Chinese pace. The research on several surgical specialties has been summarized at the very first time. AR will gradually to be more involved and take important role of the research.
... Of the included studies, most studies were either randomized controlled trials (RCTs; 47/114, 41.2%) or other experimental design studies (eg, before-and-after and cross-over studies; 49/114, 42.9%). Of the 114 studies, 14 (12.3%) were cross-sectional studies [35][36][37][38][39][40][41][42][43][44][45][46][47][48][49], 3 (2.6%) were case series or case studies [42,50,51], and 1 (0.9%) was a meta-analysis that examined the effectiveness of 3D anatomical models in teaching anatomy [52], which found that 3D anatomical models yielded significantly better results for user satisfaction and perceived effectiveness compared with conventional 2D teaching methods. An overview of the study characteristics is provided in Table 1. ...
... Among the 96 RCTs and experimental studies included, 50 (52%) compared VR against a traditional learning method (eg, box trainer and video-based lectures), 27 (28%) evaluated VR modalities by changing another variable (eg, VR vs VR with warm-up and VR with guidance vs no guidance) [9,14,48,49,, 14 (15%) did not have any intervention (eg, before-and-after studies and learning curves) [95][96][97][98][99][100][101][102][103][104][105][106][107][108][109], and 5 (5%) compared a VR modality against another type of VR modality (eg, LapSim vs ProMIS) [110][111][112][113]. ...
... Of the 114 papers, 69 (60.5%) concerned surgical VR simulators [36,37,42,46,47,51,53,[55][56][57]59 [149]. There were also a notable number of studies using ARTHRO Mentor [150] (7/69, 10%) [36,56,66,110,112,120,121], Eyesi Virtual Simulator (3/69, 4%) [37,51,133], da Vinci Surgical Simulator (4/69, 6%) [90,96,117,118], dV-Trainer (4/69, 6%) [82,88,103,126], VBLaST suturing simulator (3/69, 4%) [49,84,99], and SimSurgery (3/69, 4%) [106,125,131]. Other surgical VR simulators were uncommon. ...
Background:
Virtual reality (VR) produces a virtual manifestation of the real world and has been shown to be useful as a digital education modality. As VR encompasses different modalities, tools, and applications, there is a need to explore how VR has been used in medical education.
Objective:
The objective of this scoping review is to map existing research on the use of VR in undergraduate medical education and to identify areas of future research.
Methods:
We performed a search of 4 bibliographic databases in December 2020. Data were extracted using a standardized data extraction form. The study was conducted according to the Joanna Briggs Institute methodology for scoping reviews and reported in line with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines.
Results:
Of the 114 included studies, 69 (60.5%) reported the use of commercially available surgical VR simulators. Other VR modalities included 3D models (15/114, 13.2%) and virtual worlds (20/114, 17.5%), which were mainly used for anatomy education. Most of the VR modalities included were semi-immersive (68/114, 59.6%) and were of high interactivity (79/114, 69.3%). There is limited evidence on the use of more novel VR modalities, such as mobile VR and virtual dissection tables (8/114, 7%), as well as the use of VR for nonsurgical and nonpsychomotor skills training (20/114, 17.5%) or in a group setting (16/114, 14%). Only 2.6% (3/114) of the studies reported the use of conceptual frameworks or theories in the design of VR.
Conclusions:
Despite the extensive research available on VR in medical education, there continue to be important gaps in the evidence. Future studies should explore the use of VR for the development of nonpsychomotor skills and in areas other than surgery and anatomy.
International registered report identifier (irrid):
RR2-10.1136/bmjopen-2020-046986.
... These expert proficiency scores are as follows: Peg Transfer, 48 s with no drops outside the field of view; Extracorporeal Suture, 98 s with suture within 1 mm of target and secure knot; and Intracorporeal Suture, 89 s with suture within 1 mm of target and secure knot. Ten task repetitions and approximately 5 h of training are targets that have been identified in learning curve studies at which improvement in performance may be observed [15][16][17]. A target of 10 repetitions per task was therefore used as a practical cutoff point in the event that expert proficiency level could not be reached. ...
Introduction
The benefits of minimally invasive surgery are numerous; however, considerable variability exists in its application and there is a lack of standardized training for important advanced skills. Our goal was to determine whether participation in an advanced laparoscopic curriculum (ALC) results in improved laparoscopic suturing skills.
Methods and procedures
Study design was a prospective, randomized controlled trial. Surgery novices and trainees underwent baseline FLS training and were pre-tested on bench models. Participants were stratified by pre-test score and randomized to undergo either further FLS training (control group) or ALC training (intervention group). All were post-tested on the same bench model. Tests for differences between post-test scores of cohorts were performed using least squared means. Multivariable regression identified predictors of post-test score, and Wilcoxon rank sum test assessed for differences in confidence improvement in laparoscopic suturing ability between groups.
Results
Between November 2018 and May 2019, 25 participants completed the study (16 females; 9 males). After adjustment for relevant variables, participants randomized to the ALC group had significantly higher post-test scores than those undergoing FLS training alone (mean score 90.50 versus 82.99, p = 0.001). The only demographic or other variables found to predict post-test score include level of training (p = 0.049) and reported years of video gaming (p = 0.034). There was no difference in confidence improvement between groups.
Conclusions
Training using the ALC as opposed to basic laparoscopic skills training only is associated with superior advanced laparoscopic suturing performance without affecting improvement in reported confidence levels. Performance on advanced laparoscopic suturing tasks may be predicted by lifetime cumulative video gaming history and year of training but does not appear to be associated with other factors previously studied in relation to basic laparoscopic skills, such as surgical career aspiration or musical ability.
... The development of a CUSUM analysis requires a task to have binary outcomes (e.g., pass or fail) in order to retain objectivity. By plotting consecutive operative outcomes on a CUSUM chart, surgeons are able to view their learning curve for a given task [17,18]. ...
Background
Current evaluation methods for robotic-assisted surgery (ARCS or GEARS) are limited to 5-point Likert scales which are inherently time-consuming and require a degree of subjective scoring. In this study, we demonstrate a method to break down complex robotic surgical procedures using a combination of an objective cumulative sum (CUSUM) analysis and kinematics data obtained from the da Vinci® Surgical System to evaluate the performance of novice robotic surgeons.
Methods
Two HPB fellows performed 40 robotic-assisted hepaticojejunostomy reconstructions to model a portion of a Whipple procedure. Kinematics data from the da Vinci® system was recorded using the dV Logger® while CUSUM analyses were performed for each procedural step. Each kinematic variable was modeled using machine learning to reflect the fellows’ learning curves for each task. Statistically significant kinematics variables were then combined into a single formula to create the operative robotic index (ORI).
Results
The inflection points of our overall CUSUM analysis showed improvement in technical performance beginning at trial 16. The derived ORI model showed a strong fit to our observed kinematics data (R² = 0.796) with an ability to distinguish between novice and intermediate robotic performance with 89.3% overall accuracy.
Conclusions
In this study, we demonstrate a novel approach to objectively break down novice performance on the da Vinci® Surgical System. We identified kinematics variables associated with improved overall technical performance to create an objective ORI. This approach to robotic operative evaluation demonstrates a valuable method to break down complex surgical procedures in an objective, stepwise fashion. Continued research into objective methods of evaluation for robotic surgery will be invaluable for future training and clinical implementation of the robotic platform.
... 9,44,45 As a representation of a virtual surgical trainer, we utilized the VBLaST, a virtual reality-based simulator that replicates the FLS training tasks. 25,[46][47][48][49][50] We employed a commercially available fNIRS system to measure functional brain activation during surgical training pattern cutting tasks (CW6 system, TechEn Inc., Massachusetts). Infrared light was delivered at 690 and 830 nm to eight different sources that were coupled to eight different short separation detectors and 16 long separation detectors. ...
Significance: Surgical simulators, both virtual and physical, are increasingly used as training tools for teaching and assessing surgical technical skills. However, the metrics used for assessment in these simulation environments are often subjective and inconsistent. Aim: We propose functional activation metrics, derived from brain imaging measurements, to objectively assess the correspondence between brain activation with surgical motor skills for subjects with varying degrees of surgical skill. Approach: Cortical activation based on changes in the oxygenated hemoglobin (HbO) of 36 subjects was measured using functional near-infrared spectroscopy at the prefrontal cortex (PFC), primary motor cortex, and supplementary motor area (SMA) due to their association with motor skill learning. Inter-regional functional connectivity metrics, namely, wavelet coherence (WCO) and wavelet phase coherence were derived from HbO changes to correlate brain activity to surgical motor skill levels objectively. Results: One-way multivariate analysis of variance found a statistically significant difference in the inter-regional WCO metrics for physical simulator based on Wilk's Λ for expert versus novice,
F
(
10,1
)
=
7495.5
,
p
<
0.01
. Partial eta squared effect size for the inter-regional WCO metrics was found to be highest between the central prefrontal cortex (CPFC) and SMA, CPFC-SMA (
η
2
=
0.257
). Two-tailed Mann-Whitney U tests with a 95% confidence interval showed baseline equivalence and a statistically significant (
p
<
0.001
) difference in the CPFC-SMA WPCO metrics for the physical simulator training group (
0.960
±
0.045
) versus the untrained control group (
0.735
±
0.177
) following training for 10 consecutive days in addition to the pretest and posttest days. Conclusion: We show that brain functional connectivity WCO metric corresponds to surgical motor skills in the laparoscopic physical simulators. Functional connectivity between the CPFC and the SMA is lower for subjects that exhibit expert surgical motor skills than untrained subjects in laparoscopic physical simulators.
... Previous studies show that a minimum of 6 participants per group is required for assessing training [24,41]. Therefore, a total of 26 participants (16 males, 10 females) were retained for the study (N = 26), and were divided into three groups: the control group (N = 9) and two training groups (N = 9 and N = 8, respectively for training with the virtual hand and the tools only representation). ...
... The cumulative summation (CUSUM) technique was initially suggested to monitor surgical performance [69] but was recently applied to analyze the learning curve of a surgical skill [61], [62], [70]- [73]. CUSUM has a simple formulation that positive or negative increments are added to a cumulative score according to the failure or success of successive trials [74]. ...
... To accomplish that, we have performed a meta-analysis of bimanual skill acquisition in pattern cutting task, which is a part of the FLS program [62], [73]. Learning data on the physical FLS trainer box as well as on a virtual basic laparoscopic skill trainer (VBLaST) replicating the FLS tasks [49], [60], [61], [203], [204] ...
... However, their protocol only involved one-day training with eight repetitions, whereas in our protocol, each participant practiced up to 10 trials from training day 2 to day 12, resulting in above 100 repetitions. Literature showed that the learners could not reach proficiency in one day, which has also been demonstrated in previous studies [48], [50], [61]- [63], [73], [102], [218]. Especially in [50], 66-161 repetitions of the FLS task were required to achieve proficiency. ...
The traditional surgical apprenticeship model of “see one, do one, teach one” that was standardized by Halsted more than a century ago is still in effect in residency programs around the world. Though, it has been greatly enhanced via the use of advanced technologies, such as training simulators as well as standardized tasks and assessment metrics. Notably, the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and the American College of Surgeons (ACS) developed the Fundamentals of Laparoscopic Surgery (FLS) program. The FLS training and assessment portions of this program are prerequisites to board certification in general surgery and hence, impact every general surgery resident in the USA. Though, the current practice relies on scoring metrics that can suffer from subjectivity and inconsistencies while personnel intensive and time-consuming. Moreover, the current training protocols rely on repetitive execution of the tasks without remediation during the training leading to more practice than needed and a plateau that is less than the maximal potential level. In this project, we aim at enhancing the methodologies to assess surgical performance as well as investigating new approaches to facilitate surgical skill performance, acquisition, retention, and transfer beyond mere task repetition.
Firstly, we propose a machine learning methodology that could predict the learning curve features from the beginning of the training, which indicates that the training protocol could be personalized. We further defined a single factor that can describe how the learning curve parameters are related to each other.
Secondly, we explore the possibility to emulate the current standardized surgical skill metric employed in the field, namely the FLS score, by combining neuroimaging data acquired during the task execution and machine learning methodologies for potentially fast and bedside implementation. In this context, we have validated a deep neural network, Brain-NET, that accurately predicts performance scores from hemodynamic data from the brain obtained using functional near-infrared spectroscopy (fNIRS). Furthermore, we are also currently implementing deep learning approaches to improve and speed up the fNIRS data preprocessing workflow towards enabling real-time implementation.
Lastly, we investigate the potentials of transcranial electrical stimulation (tES) in enhancing bimanual surgical skills. We performed a series of studies to explore the short term and long term effects of tES on surgical motor skill performance. Transcranial random noise stimulation (tRNS) enhances surgical motor skill performance in the short term, while transcranial direct current stimulation (tDCS) improves long term skill acquisition in decreasing surgical error. The concurrently acquired fNIRS data, while medical students performed an FLS task, elucidated a correlation between cortical activations as reported by fNIRS with tES excitation and improved performance.
... By analyzing the features of the learning curve on different training systems, we can compare differences in the learning process to compare and evaluate the effectiveness of the systems in terms of training. The cumulative summation (CUSUM) is a criterion-based evaluation of the learning process [24,25]. It can be used to analyze the performance over time within a set of criteria. ...
... It can be used to analyze the performance over time within a set of criteria. Previous studies have assessed the learning curve of peg transfer and pattern cutting tasks of the VBLaST© simulator [23,25,26]. The purpose of this study was, by constructing the CUSUM learning curves for participants' performance on both systems, to evaluate the training effectiveness of the VBLaST-SS© and to assess whether the skills are retained after 2 weeks without training. ...
... Fourteen medical students from 1st to 3rd year (8 male and 6 female) participated in this study. The average age was 23.4 years (range [21][22][23][24][25][26][27]. All participants were novices in intracorporeal suturing at the start of the study and did not have prior exposure to either the FLS box trainer or the VBLaST-SS© simulator. ...
Background
The virtual basic laparoscopic skill trainer suturing simulator (VBLaST-SS©) was developed to simulate the intracorporeal suturing task in the FLS program. The purpose of this study was to evaluate the training effectiveness and participants’ learning curves on the VBLaST-SS© and to assess whether the skills were retained after 2 weeks without training.
Methods
Fourteen medical students participated in the study. Participants were randomly assigned to two training groups (7 per group): VBLaST-SS© or FLS, based on the modality of training. Participants practiced on their assigned system for one session (30 min or up to ten repetitions) a day, 5 days a week for three consecutive weeks. Their baseline, post-test, and retention (after 2 weeks) performance were also analyzed. Participants’ performance scores were calculated based on the original FLS scoring system. The cumulative summation (CUSUM) method was used to evaluate learning. Two-way mixed factorial ANOVA was used to compare the effects of group, time point (baseline, post-test, and retention), and their interaction on performance.
Results
Six out of seven participants in each group reached the predefined proficiency level after 7 days of training. Participants’ performance improved significantly (p < 0.001) after training within their assigned group. The CUSUM learning curve shows that one participant in each group achieved 5% failure rate by the end of the training period. Twelve out of fourteen participants’ CUSUM curves showed a negative trend toward achieving the 5% failure rate after further training.
Conclusion
The VBLaST-SS© is effective in training laparoscopic suturing skill. Participants’ performance of intracorporeal suturing was significantly improved after training on both systems and was retained after 2 weeks of no training.
... We utilize the official FLS box trainer as the physical simulator since it is widely used for training laparoscopic skills and is validated for board certification [8,30,31]. We use the validated Virtual Basic Laparoscopic Skills Trainer (VBLaST) system, which replicates the FLS pattern cutting task on a computer model with high fidelity [32][33][34][35][36][37], as the virtual simulator. To perform real-time brain imaging, a fNIRS system (CW6 system, TechEn Inc., MA, USA) was used to deliver infrared light. ...
Background
Physical and virtual surgical simulators are increasingly being used in training technical surgical skills. However, metrics such as completion time or subjective performance checklists often show poor correlation to transfer of skills into clinical settings. We hypothesize that non-invasive brain imaging can objectively differentiate and classify surgical skill transfer, with higher accuracy than established metrics, for subjects based on motor skill levels.
Study design
18 medical students at University at Buffalo were randomly assigned into control, physical surgical trainer, or virtual trainer groups. Training groups practiced a surgical technical task on respective simulators for 12 consecutive days. To measure skill transfer post-training, all subjects performed the technical task in an ex-vivo environment. Cortical activation was measured using functional near-infrared spectroscopy (fNIRS) in the prefrontal cortex, primary motor cortex, and supplementary motor area, due to their direct impact on motor skill learning.
Results
Classification between simulator trained and untrained subjects based on traditional metrics is poor, where misclassification errors range from 20 to 41%. Conversely, fNIRS metrics can successfully classify physical or virtual trained subjects from untrained subjects with misclassification errors of 2.2% and 8.9%, respectively. More importantly, untrained subjects are successfully classified from physical or virtual simulator trained subjects with misclassification errors of 2.7% and 9.1%, respectively.
Conclusion
fNIRS metrics are significantly more accurate than current established metrics in classifying different levels of surgical motor skill transfer. Our approach brings robustness, objectivity, and accuracy in validating the effectiveness of future surgical trainers in translating surgical skills to clinically relevant environments.
