Nigel W John

• Professor at University of Chester

We present the results of a double-blind phase 2b randomized control trial that used a custom built virtual reality environment for the cognitive rehabilitation of stroke survivors. A stroke causes damage to the brain and problem solving, memory and task sequencing are commonly affected. The brain can recover to some extent, however, and stroke pat...

Stroke recovery often depends on cortical reorganization resulting from neuroplasticity, which is highly dependent on immediate, targeted and intensive therapy poststroke. However, patients spend over 80% of their daytime in nontherapeutic activities. Despite the National Institute of Health and Social Care recommendation in the UK for 45 minutes r...

Virtual reality entertainment and serious games popularity has continued to rise but the processes for level design for VR games has not been adequately researched. Our paper contributes LevelEd VR; a generic runtime virtual reality level editor that supports the level design workflow used by developers and can potentially support user generated co...

We present VRIA, a Web-based framework for creating Immersive Analytics (IA) experiences in Virtual Reality. VRIA is built upon WebVR, A-Frame, React and D3.js, and offers a visualization creation workflow which enables users, of different levels of expertise, to rapidly develop Immersive Analytics experiences for the Web. The use of these open-sta...

As the healthcare enterprise is adopting novel imaging and health-assessment technologies, we are facing unprecedented requirements in information sharing, patient empowerment, and care coordination within the system. Medical experts not only within US, but around the world should be empowered through collaboration capabilities on 3D data to enable...

Surgical smoke removal algorithms can improve the quality of intra-operative imaging and reduce hazards in image-guided surgery, a highly desirable post-process for many clinical applications. These algorithms also enable effective computer vision tasks for future robotic surgery. In this paper, we present a new unsupervised learning framework for...

Mixed reality (MR) is a powerful interactive technology for new types of user experience. We present a semantic‐based interactive MR framework that is beyond current geometry‐based approaches, offering a step change in generating high‐level context‐aware interactions. Our key insight is that by building semantic understanding in MR, we can develop...

With the rise in popularity of serious games there is an increasing demand for virtual environments based on real-world locations. Emergency evacuation or fire safety training are prime examples of serious games that would benefit from accurate location depiction together with any application involving personal space. However, creating digital indo...

We developed an application that makes indoor modelling accessible by utilizing consumer grade technology in the form of Apple’s ARKit and a smartphone to assist with serious games level design. We compared our system to that of a tape measure and a system based on an infra-red depth sensor and application. We evaluated the accuracy and efficiency...

We report on the design, implementation and evaluation of , a framework for building immersive analytics (IA) solutions in Web-based Virtual Reality (VR), built upon WebVR, A-Frame, React and D3. The recent emergence of affordable VR interfaces have reignited the interest of researchers and developers in exploring new, immersive ways to visualize d...

Convolutional Neural Networks (CNNs) need large amounts of data with ground truth annotation, which is a challenging problem that has limited the development and fast deployment of CNNs for many computer vision tasks. We propose a novel framework for depth estimation from monocular images with corresponding confidence in a self-supervised manner. A...

Mixed Reality (MR) is a powerful interactive technology that yields new types of user experience. We present a semantic based interactive MR framework that exceeds the current geometry level approaches, a step change in generating high-level context-aware interactions. Our key insight is to build semantic understanding in MR that not only can great...

Background and objective: While Minimally Invasive Surgery (MIS) offers considerable benefits to patients, it also imposes big challenges on a surgeon’s performance due to well-known issues and restrictions associated with the field of view (FOV), hand-eye misalignment and disorientation, as well as the lack of stereoscopic depth perception in mono...

We have developed an augmented reality tool for radiotherapy to view the real world scene, i.e. the patient on a treatment couch, combined with computer graphics content, such as planning image data and any defined outlines of organ structures. We have deployed our software to a number of consumer electronics devices (iPad, Android tablets, MS Holo...

The potential of Augmented Reality (AR) technology to assist minimally invasive surgeries (MIS) lies in its computational performance and accuracy in dealing with challenging MIS scenes. Even with the latest hardware and software technologies, achieving both real-time and accurate augmented information overlay in MIS is still a formidable task. In...

Mixed Reality (MR) is of increasing interest within technology-driven modern medicine but is not yet used in everyday practice. This situation is changing rapidly, however, and this paper explores the emergence of MR technology and the importance of its utility within medical applications. A classification of medical MR has been obtained by applyin...

Mixed Reality (MR) is of increasing interest within technology-driven modern medicine but is not yet used in everyday practice. This situation is changing rapidly, however, and this paper explores the emergence of MR technology and the importance of its utility within medical applications. A classification of medical MR has been obtained by applyin...

Navigating a powered wheelchair and avoiding collisions is often a daunting task for new wheelchair users. It takes time and practice to gain the coordination needed to become a competent driver and this can be even more of a challenge for someone with a disability. We present a cost-effective virtual reality (VR) application that takes advantage o...

We have previously investigated the effectiveness of a custom-built virtual environment in assisting training of a ventriculostomy procedure, which is a commonly performed procedure by a neurosurgeon and a core task for trainee surgeons. The training tool (called VCath) was initially developed as a low-fidelity app for a tablet platform to provide...

One of the major challenges in Minimally Invasive Surgery (MIS) such as laparoscopy is the lack of depth perception. In recent years, laparoscopic scene tracking and surface reconstruction has been a focus of investigation to provide rich additional information to aid the surgical process and compensate for the depth perception issue. However, robu...

Control of a powered wheelchair is often not intuitive, making training of new users a challenging and sometimes hazardous task. Collisions, due to a lack of experience can result in injury for the user and other individuals. By conducting training activities in virtual reality (VR), we can potentially improve driving skills whilst avoiding the ris...

The use of electric wheelchairs is inherently risky, as collisions due to lack of control can result in injury for the user, but also potentially for other pedestrians. Introducing new users to powered chairs via virtual reality (VR) provides one possible solution, as it eliminates the risks inherent to the real world during training. However, trad...

The use of electric wheelchairs is inherently risky, as collisions due to lack of control can result in injury for the user, but also potentially for other pedestrians. Introducing new users to powered chairs via virtual reality (VR) provides one possible solution, as it eliminates the risks inherent to the real world during training. However, trad...

The use of electric wheelchairs is inherently risky, as collisions due to lack of control can result in injury for the user, but also potentially for other pedestrians. Introducing new users to powered chairs via virtual reality (VR) provides one possible solution, as it eliminates the risks inherent to the real world during training. However, trad...

The use of electric wheelchairs is inherently risky, as collisions due to lack of control can result in injury for the user, but also potentially for other pedestrians. Introducing new users to powered chairs via virtual reality (VR) provides one possible solution, as it eliminates the risks inherent to the real world during training. However, trad...

We present the development of a transperineal prostate biopsy, with high fidelity haptic feedback. We describe our current prototype, which is using physical props and a Geomagic Touch. In addition, we discuss a method for collecting in vitro axial needle forces, for programming haptic feedback, along with implemented an forthcoming features such a...

Two-dimensional asymmetry, border irregularity, colour variegation and diameter (ABCD) features are important indicators currently used for computer-assisted diagnosis of malignant melanoma (MM); however, they often prove to be insufficient to make a convincing diagnosis. Previous work has demonstrated that 3D skin surface normal features in the fo...

The requirement for training surgical procedures without exposing the patient to additional risk is well accepted and is part of a national drive in the UK and internationally. Computer-based simulations are important in this context, including neurosurgical resident training. The objective of this study is to evaluate the effectiveness of a custom...

We have developed a system called UltraSendo that creates a force field in space using an array of ultrasonic transducers cooperatively emitting ultrasonic waves to a focal point. UltraSendo is the first application of this technology in the context of medical training simulators. A face validation study was carried out at a Catheter Laboratory in...

We have designed a configurable virtual environment to train rugby ball passing skills. Seeking to validate the system’s ability to correctly aid training, two experiments were performed. Ten participants took part in ball passing activities, which were used to compare the combinations of different user positions relative to the physical screen, th...

We have designed and developed a virtual environment to train rugby ball passing skills. Seeking to validate the system's ability to correctly aid training, an initial experiment was performed to examine the effect of stereoscopic technology and the physical screen's setup on the user's ability to perceive virtual distances correctly. Four particip...

Background Medical mentoring is becoming increasingly complex with the evolving needs of trainees and the complexities of their personal and social lives. The Internet is an enabling technology, which increasingly facilitates interaction with multiple people at a distance. Web 2.0 and 3.0 technology shows promise in furthering this facilitation. Ob...

Purpose We review augmented (AR) and virtual reality (VR) applications in radiotherapy as found in the scientific literature and highlight future developments enabled by the use of small mass-produced devices and portability of techniques developed in other fields to radiotherapy. Analysis The application of AR and VR within radiotherapy is still i...

Training in Interventional Radiology currently uses the apprenticeship model, where clinical and technical skills of invasive procedures are learnt during practice in patients. This apprenticeship training method is increasingly limited by regulatory restrictions on working hours, concerns over patient risk through trainees' inexperience and the va...

Medical simulators provide a risk-free environment for trainee doctors to practice and improve their skills. UltraPulse is a new tactile system designed to utilise focussed airborne ultrasound to mimic a pulsation effect such as that of a human arterial pulse. In this paper, we focus on the construction of the haptics component, which can later be...

Visual computing represents one of the most challenging and inspiring arenas in computer science. Today, fifty percent of content on the internet is in the form of visual data and information, and more than fifty percent of the neurons in the human brain are used in visual perception and reasoning. RIVIC is the collaborative amalgamation of researc...

Augmented Reality technology has been used in medical visualization applications in various different ways. Haptics, on the other hand, are a popular method of interacting in Augmented and Virtual Reality environments. We present how reliance on standards benefits the fusion of these technologies, through a series of research themes, carried out in...

VCath is a neurosurgery training tool for the catheterization of the lateral ventricle that has been designed for use on tablet devices. We believe this is the first use of a tablet (iPad) for this purpose and demonstrates future utility for this approach, particularly for Objective Structured Clinical Exams (OSCEs). This paper outlines the impleme...

Osteoarthritis of the hip is commonly caused by the repetitive contact between abnormal skeletal prominences between the anterosuperior femoral head-neck junction and the rim of the acetabular socket. Current methods for estimating femoroacetabular impingement by analyzing the sphericity of the femoral head require manual measurements which are bot...

This paper introduces a novel technique for the visualization of blood (or other fluid) flowing through a complex 3D network of vessels. The Directed Particle System (DPS) approach is loosely based on the computer graphics concept of flocking agents. It has been developed and optimised to provide effective real time visualization and qualitative si...

Mentoring, for physicians and surgeons in training, is advocated as an essential adjunct in work-based learning, providing support in career and non-career related issues. The World Wide Web (WWW) has evolved, as a technology, to become more interactive and person centric, tailoring itself to the individual needs of the user. This changing technolo...

Medical simulators provide a risk-free environment for trainee doctors to practice and improve their skills. UltraPulse is a new tactile system designed to utilise focussed airborne ultrasound to mimic a pulsation effect such as that of a human arterial pulse. In this paper, we focus on the construction of the haptics component, which can later be...

Interventional Radiology procedures (e.g., angioplasty, embolization, stent graft placement) provide minimally invasive therapy to treat a wide range of conditions. These procedures involve the use of flexible tipped guidewires to advance diagnostic or therapeutic catheters into a patient's vascular or visceral anatomy. This paper presents a real-t...

We propose a method to automatically tune a patient-based virtual environment training simulator for abdominal needle insertion. The key attributes to be customized in our framework are the elasticity of soft-tissues and the respiratory model parameters. The estimation is based on two 3D Computed Tomography (CT) scans of the same patient at two dif...

A proposed method visualizes the surface appearance of living human brain tissue. The goal is to investigate whether realistic models of living anatomy are possible and, if so, whether they provide added value to anatomy education and training simulators. From calibrated photography of exposed brain tissue and suitable alternatives, experiments pro...

This paper presents a virtual environment for training femoral palpation and needle insertion, the opening steps of many interventional radiology procedures. A novel augmented reality simulation called PalpSim has been developed that allows the trainees to feel a virtual patient using their own hands. The palpation step requires both force and tact...

Guidewire insertion is an imperative task of minimally invasive medical procedures. During the procedure, surgeons need to steer long flexible thin wires through patient's blood vessels to reach a clinical target. In this paper, we present a novel approach to model haptics of guidewire insertion process for training simulation. The algorithm also a...

This review paper discusses the role of haptics within virtual medical training applications, particularly, where it can be used to aid a practitioner to learn and practice a task. The review summarizes aspects to be considered in the deployment of haptics technologies in medical training. First, both force/torque and tactile feedback hardware solu...

This paper presents a virtual environment for training femoral palpation and needle insertion, the opening steps of many interventional radiology procedures. A novel augmented reality simulation called PalpSim has been developed that allows the trainees to feel a virtual patient using their own hands. The palpation step requires both force and tact...

The Virtual Environment Rugby Skills Trainer (VERST) is an on-going collaborative project between Bangor University's Schools of Computer Science, Sports Health and Education Sciences and Psychology, together with Rugby Innovations Ltd., to create an accessible training tool that will help professional rugby players improve certain skills. It is sa...

The Virtual Environment Rugby Skills Trainer (VERST) is an on-going collaborative project between Bangor University?s Schools of Computer Science, Sports Health and Education Sciences and Psychology, together with Rugby Innovations Ltd., to create an accessible training tool that will help professional rugby players improve certain skills. It is sa...

This paper presents an advanced method of visualizing the surface appearance of living brain tissue. We have been granted access to the operating theatre during neurosurgical procedures to obtain colour data via calibrated photography of exposed brain tissue. The specular reflectivity of the brain's surface is approximated by analyzing a gelatine l...

A SensAble Omni force feedback device has been modified to increase the face validity of a needle insertion simulation. The new end effector uses a real needle hub and shortened needle shaft in place of the Omni's pre-fitted pen shaped end effector. This modification facilitates correct procedural training through the simulation of co-located visua...

Within the limits of current technology, many applications of a virtual environment will trade-off accuracy for speed. This is not an acceptable compromise in a medical training application where both are essential. Efficient algorithms must therefore be developed. The purpose of this project is the development and validation of a novel physics-bas...

In this paper we summarize the progress of the Web3D scene graph model, and associated standards, specifically Extensible 3D (X3D) in the domain of medical simulation. Historically, the Web3D nodesets have focused on the representation and rendering of point, line or surface geometry. More recently, significant progress in X3D Volume rendering has...

Purpose: We previously proposed to compute the X-ray attenuation from polygons directly on the GPU, using OpenGL, to significantly increase performance without loss of accuracy. The method has been deployed into a training simulator for percutaneous transhepatic cholangiography. The simulations were however restricted to monochromatic X-rays using...

Purpose: We previously proposed to compute the X-ray attenuation from polygons directly on the GPU, using OpenGL, to significantly increase performance without loss of accuracy. The method has been deployed into a training simulator for percutaneous transhepatic cholangiography. The simulations were however restricted to monochromatic X-rays using...

In this paper, we propose a deterministic simulation of X-ray transmission imaging on graphics hardware. Only the directly transmitted photons are simulated, using the Beer-Lambert law. Our previous attempt to simulate Xray attenuation from polygon meshes utilising the GPU showed significant increase of performance, with respect to a validated soft...

Haptics technologies are frequently used in virtual environments to allow participants to touch virtual objects. Medical applications are no exception and a wide variety of commercial and bespoke haptics hardware solutions have been employed to aid in the simulation of medical procedures. Intuitively the use of haptics will improve the training of...

The use of Virtual Environments has been widely reported as a method of teaching anatomy. Generally such environments only convey the shape of the anatomy to the student. We present the Bangor Augmented Reality Education Tool for Anatomy (BARETA), a system that combines Augmented Reality (AR) technology with models produced using Rapid Prototyping...

A needle insertion is a widely performed procedure used either to inject fluids, to retrieve samples or as an introducing conduit for more advanced procedures.Needle insertions, like most medical procedures, pose an inherent risk of complication to the patient. This risk has prompted the development of a variety of haptic training simulators to aid...

A femoral palpation simulation for training purposes has been developed to simulate the initial steps of the Seldinger technique which is currently neglected in both commercial and academic medical training simulations. The simulation co-locates visual and haptic feedback through the use of an augmented reality video see-through visualisation whils...

An overview to medical simulation has been provided. In the context of procedural interventional radiology training, we start with the definition and history of simulation, address its increasing importance in medicine reflect on its theoretical basis and current evidence and finally review its advantages/ limitations and prospects for the future.

Purpose We present here a simulator for interventional radiology focusing on percutaneous transhepatic cholangiography (PTC). This procedure consists of inserting a needle into the biliary tree using fluoroscopy for guidance. Methods The requirements of the simulator have been driven by a task analysis. The three main components have been identifie...

Commercial interventional radiology vascular simulators emulate instrument navigation and device deployment, though none supports the Seldinger technique, which provides initial access to the vascular tree. This paper presents a novel virtual environment for teaching this core skill. Our simulator combines two haptic devices: vessel puncture with a...

AIM: To determine face and content validity of a physics based virtual reality interventional radiology simulator, Imagine-S (Imaging guided interventional needle puncture simulation).

Haptics force-feedback technology is fast becoming a consumer product and no longer only found in research laboratories. The emergence of the budget Falcon device (Novint Technologies, Inc., USA) represents a key step in the dissemination of haptics technology as it offers this functionality to home users, in particular to games players. Haptics ha...

Two different approaches for the preparation of novel cost-effective molecular haptic applications (Figure 1) are described. The former utilises Perl scripting within a commercial molecular modelling package to generate static / animated H3D scene graphs for haptic CPK space-filling atomic perception. Within the second approach, key chemical concep...

Medical visualization in a hospital can be used to aid training, diagnosis, and pre- and intra-operative planning. In such an application, a virtual representation of a patient is needed that is interactive, can be viewed in three dimensions (3D), and simulates physiological processes that change over time. This paper highlights some of the computa...

We present an integrated system for training visceral needle puncture procedures. Our aim is to provide a cost effective and validated training tool that uses actual patient data to enable interventional radiology trainees to learn how to carry out image-guided needle puncture. The input data required is a computed tomography scan of the patient th...

We present an integrated system for training visceral needle puncture procedures. Our aim is to provide a cost effective and validated training tool that uses actual patient data to enable interventional radiology trainees to learn how to carry out image-guided needle puncture. The input data required is a computed tomography scan of the patient th...

Interventional radiology is a rapidly expanding speciality using minimally invasive techniques to treat a multitude of clinical problems. Current work in progress aims to create an affordable virtual training tool to reduce training times and patient risk during a trainee practitioners learning cycle. The procedure of arterial catheterisation has b...

This paper reports on a low cost system for training ultrasound imaging techniques. The need for such training is particularly acute in developing countries where typically ultrasound scanners remain idle due to the lack of experienced sonographers. The system described below is aimed at a PC platform but uses interface components from the Nintendo...

We present an integrated system for training ultrasound guided needle puncture. Our aim is to provide a cost effective and validated training tool that uses actual patient data to enable interventional radiology trainees to learn how to carry out image-guided needle puncture. The input data required is a computed tomography scan of the patient that...

In this paper, we propose to take advantage of computer graphics hardware to achieve an accelerated simulation of X-ray transmission imaging, and we compare results with a fast and robust software-only implementation. The running times of the GPU and CPU implementations are compared in different test cases. The results show that the GPU implementat...

Novel, simple, cost-effective applications combining haptics and computer graphics for the study of key chemical concepts such as reactivity and periodicity at AS/A-level and undergraduate level are described.

Interventional Radiology is a rapidly expanding speciality using minimally invasive techniques to treat a multitude of clinical problems. Current work in progress aims to create an affordable virtual training tool to reduce training times and patient risk during a trainee practitioners learning cycle. The procedure of arterial catheterisation has b...

Throughout the practice of procedural medicine, there is an unrelenting shift to management by less invasive techniques such as interventional radiology (IR). This subspecialty within radiology uses imaging to guide needles, wires and catheters using tiny access incisions. Like other minimally invasive techniques, risk, pain and recovery times are...

This paper discusses the design issues and implementation details of building a medical training simulator. Example projects that have been undertaken by the Visualization and Medical Graphics group at Bangor University and our collaborators are used to illustrate the points made. A detailed case study is then presented of a virtual environment des...

Introduction and Motivations We present a method for modelling the force penetration of needles into anatomic structures that are encountered during visceral punctures. Our aim is to provide a validated haptic model that can be used for the insertion of needles within our developing medical simulations of visceral interventional needle puncture pro...

PURPOSE-MATERIALS: To use patient imaging as the basis for developing virtual environments (VE). BACKGROUND Interventional radiology basic skills are still taught in an apprenticeship in patients, though these could be learnt in high fidelity simulations using VE. Ideally, imaging data sets for simulation of image-guided procedures would alter dyna...

We present an integrated system for training ultrasound (US) guided needle puncture. Our aim is to provide a validated training tool for interventional radiology (IR) that uses actual patient data. IR procedures are highly reliant on the sense of touch and so haptic hardware is an important part of our solution. A hybrid surface/volume haptic rende...