Xuesong Ye’s research while affiliated with Zhejiang University and other places

Background Robot and navigation systems can relieve surgeon's difficulties in delicate and safe operation in tortuous lumens in traditional intraluminal procedures (IP). This paper aims to review the three key components of these systems: registration, path planning and shape reconstruction and highlight their limitations and future perspectives. Methods An electronic search for relevant studies was performed in Web of Science and Google scholar databases until 2024. Results As for 2D–3D registration in IP, we focused on analysing feature extraction. For path planning, this paper proposed a new classification method and focused on selection of planning space and the establishment of path cost. Regarding shape reconstruction, the pros and cons of existing methods are analysed and methods based on fibre optic sensors and electromagnetic (EM) tracking are focused on. Conclusion These three technologies in IP have made great progress, but there are still challenges that require further research.

Background In robot‐assisted flexible ureteroscopy, planning a safety‐enhanced path facilitates ureteroscope reaching the target safely and quickly. However, current methods rarely consider the safety impact caused by body motion of the anterior part, such as impingement on the lumen wall and sweeping motion risk, or the method can only be used in collision‐free situations. Methods The kinematic model of the anterior part under C‐shaped and S‐shaped collision bending is first analysed. Considering the newly defined impingement cost and sweeping area, the differential evolution algorithm is adopted to optimise the path in the configuration space. Experiments were performed to verify the effectiveness of the method. Results Compared with the competing algorithm, the proposed algorithm reduced impingement cost and sweeping area by an average of 31.0% and 8.64%. Force measurement experiments verified the rationality of the impingement cost expression. Conclusion The experimental results proved the feasibility of the proposed path planning algorithm.

Stereo matching is a crucial technology in the binocular laparoscopic-based surgical navigation systems. In recent years, neural networks have been widely applied to stereo matching and demonstrated outstanding performance. however, this method heavily relies on manual feature engineering meaning that professionals must be involved in the feature extraction and matching. This process is both time-consuming and demands specific expertise. This paper introduces a novel stereo matching framework DCStereo that realizes a fully automatic neural architecture design for the stereo matching of binocular laparoscopic images. The proposed framework utilizes a densely connected search space which enables a more flexible and diverse architecture composition. Furthermore, the proposed algorithm leverages the channel and path sampling strategies to reduce memory consumption during searching. Empirically, our searched DCStereo on the SCARED training dataset achieves a mean absolute error of 3.589 mm on the test dataset, which outperforms hand-crafted stereo matching methods and other approaches. Furthermore, when directly testing on the SERV-CT dataset, our DCStereo demonstrates better generalization ability than other methods. Our proposed approach leverages the neural architecture search technique and a densely connected search space for automatic neural architecture design in stereo matching of binocular laparoscopic images. Our method delivers advanced performance on the SCARED dataset and promising results on the SERV-CT dataset. These findings demonstrate the potential of our approach for improving clinical surgical navigation systems.

Colorectal cancer is a leading cause of cancer-related deaths globally. In recent years, the use of convolutional neural networks in computer-aided diagnosis (CAD) has facilitated simpler detection of early lesions like polyps during real-time colonoscopy. However, the majority of existing techniques require a large training dataset annotated by experienced experts. To alleviate the laborious task of image annotation and utilize the vast amounts of readily available unlabeled colonoscopy data to further improve the polyp detection ability, this study proposed a novel self-supervised representation learning method called feature pyramid siamese networks (FPSiam). First, a feature pyramid encoder module was proposed to effectively extract and fuse both local and global feature representations among colonoscopic images, which is important for dense prediction tasks like polyp detection. Next, a self-supervised visual feature representation containing the general feature of colonoscopic images is learned by the siamese networks. Finally, the feature representation will be transferred to the downstream colorectal polyp detection task. A total of 103 videos (861,400 frames), 100 videos (24,789 frames), and 60 videos (15,397 frames) in the LDPolypVideo dataset are used to pre-train, train, and test the performance of the proposed FPSiam and its counterparts, respectively. The experimental results have illustrated that our FPSiam approach obtains the optimal capability, which is better than that of other state-of-the-art self-supervised learning methods and is also higher than the method based on transfer learning by 2.3 mAP and 3.6 mAP for two typical detectors. In conclusion, FPSiam provides a cost-efficient solution for developing colorectal polyp detection systems, especially in conditions where only a small fraction of the dataset is labeled while the majority remains unlabeled. Besides, it also brings fresh perspectives into other endoscopic image analysis tasks.

Retrograde intrarenal surgery (RIRS) is a widely utilized diagnostic and therapeutic tool for multiple upper urinary tract pathologies. The image-guided navigation system can assist the surgeon to perform precise surgery by providing the relative position between the lesion and the instrument after the intraoperative image is registered with the preoperative model. However, due to the structural complexity and diversity of multi-branched organs such as kidneys, bronchi, etc., the consistency of the intensity distribution of virtual and real images will be challenged, which makes the classical pure intensity registration method prone to bias and random results in a wide search domain. In this paper, we propose a structural feature similarity-based method combined with a semantic style transfer network, which significantly improves the registration accuracy when the initial state deviation is obvious. Furthermore, multi-view constraints are introduced to compensate for the collapse of spatial depth information and improve the robustness of the algorithm. Experimental studies were conducted on two models generated from patient data to evaluate the performance of the method and competing algorithms. The proposed method obtains mean target error (mTRE) of 0.971 ± 0.585 mm and 1.266 ± 0.416 mm respectively, with better accuracy and robustness overall. Experimental results demonstrate that the proposed method has the potential to be applied to RIRS and extended to other organs with similar structures.

The continuous estimation of arterial blood pressure (ABP) waveforms directly from single-channel photoplethysmography (PPG) signals will predictably change the way to monitor blood pressure in the future. This article proposed a new hybrid mathematical model for continuous blood pressure monitoring by investigating the relationship between the finger PPG signal and the radial ABP signal based on a public database. Considering potential damping factors and wave propagation/reflection in blood circulation, we combined the electrical network model with the tube-load model. The optimal range of model parameters was obtained through the system identification method to realize the individualized continuous blood pressure measurements. Compared with the invasive measurement, the hybrid model performed superior blood pressure estimation with high consistency. The estimated ABP waveforms correlated highly with the reference waveforms with an average correlation coefficient of 0.96. The mean absolute error/standard deviation of the estimated systolic blood pressure (SBP), mean arterial blood pressure (MAP), and diastolic blood pressure (DBP) were 3.0/4.4, 2.1/3.0 and 2.1/3.2 mmHg, respectively. The results met the requirements of the Association for the Advancement of Medical Instrumentation (AAMI). The hybrid model is expected to be embedded in small wearable devices to directly estimate the continuous blood pressure waveforms at the radial artery site through the PPG signals, pioneering the synchronous non-sensing monitoring on blood pressure and blood flow.

This study proposed a refined BP prediction strategy that using single-channel photoplethysmography (PPG) signals to stratify populations by cardiovascular status before BP estimation. Combining demographic characteristics (age, gender) and pulse wave morphological features, the random forest was applied to screen two kinds of typical cardiovascular diseases (CVDs), with an accuracy of 92.2%. A deep learning model (BiLSTM-At) was proposed to estimate the long-term BP trend for different CVD groups. Transfer learning technique was used for personalized modeling to reduce computational complexity while improving performance. The method was validated on 255 patients with different CVDs. The mean absolute errors (MAEs) of systolic blood pressure (SBP) and diastolic blood pressure (DBP) estimation were 2.815 mmHg and 1.876 mmHg for normal subjects, 3.024 mmHg and 1.334 mmHg for AF subjects, and 4.444 mmHg and 2.549 mmHg for CA subjects. The results met the American Association for the Advancement of Medical Instrumentations (AAMI) and British Hypertension Society (BHS) Class A criteria. This indicated that our strategy has good performance and can realize long-term monitoring of BP through a small batch samples, with the potential to implement real-time monitoring in healthy devices.

The ankle joint is one of the important joints of the human body to maintain the ability to walk. Diseases such as stroke and ankle osteoarthritis could weaken the body’s ability to control joints, causing people’s gait to be out of balance. Ankle–foot orthoses can assist users with neuro/muscular or ankle injuries to restore their natural gait. Currently, passive ankle–foot orthoses are mostly designed to fix the ankle joint and provide support for walking. With the development of materials, sensing, and control science, semi-active orthoses that release mechanical energy to assist walking when needed and can store the energy generated by body movement in elastic units, as well as active ankle–foot orthoses that use external energy to transmit enhanced torque to the ankle, have received increasing attention. This article reviews the development process of ankle–foot orthoses and proposes that the integration of new ankle–foot orthoses with rehabilitation technologies such as monitoring or myoelectric stimulation will play an important role in reducing the walking energy consumption of patients in the study of human-in-the-loop models and promoting neuro/muscular rehabilitation.