Jiahui Pan

• PhD
• Professor (Full) at South China Normal University

In this paper, a hybrid brain-computer interface (BCI) system combining P300 and steady-state visual evoked potential (SSVEP) is proposed to improve the performance of asynchronous control. Four groups of flickering buttons were set in the graphical user interface (GUI). Each group contained one large button in the center and eight small buttons ar...

Objective. The bedside detection of potential awareness in patients with disorders of consciousness (DOC) currently relies only on behavioral observations and tests; however, the misdiagnosis rates in this patient group are historically relatively high. In this study, we proposed a visual hybrid brain–computer interface (BCI) combining P300 and ste...

These authors contributed equally to this work. Cognitive motor dissociation describes a subset of patients with disorders of consciousness who show neuroimaging evidence of consciousness but no detectable command-following behaviours. Although essential for family counselling, decision-making, and the design of rehabilitation programmes, the progn...

P300 brain-computer interfaces (BCIs) have significant potential for detecting and assessing residual consciousness in patients with disorders of consciousness (DoC) but are limited by insufficient data collected from them. In this study, a multiple scale convolutional few-shot learning network (MSCNN-FSL) was proposed to detect and recognize small...

Electrocardiogram (ECG) is essential for the clinical diagnosis of arrhythmias and other heart diseases, but deep learning methods based on ECG often face limitations due to the need for high-quality annotations. Although previous ECG self-supervised learning (eSSL) methods have made significant progress in representation learning from unannotated...

Large language models (LLMs) with long-context processing are still challenging because of their implementation complexity, training efficiency and data sparsity. To address this issue, a new paradigm named Online Long-context Processing (OLP) is proposed when we process a document of unlimited length, which typically occurs in the information rece...

Objective: Attention regulation is an essential ability in daily life that affects learning and work efficiency and is closely related to mental health. The effectiveness of brain-computer interface (BCI) systems in attention regulation has been proven, but most of these systems rely on bulky and expensive equipment and are still in the experiment...

Most recent few-shot learning approaches are based on meta-learning with episodic training. However, prior studies encounter two crucial problems: (1) \textit{the presence of inductive bias}, and (2) \textit{the occurrence of catastrophic forgetting}. In this paper, we propose a novel Multi-Level Contrastive Constraints (MLCC) framework, that joint...

Numerous studies have shown that musical stimulation can activate corresponding functional brain areas. Electroencephalogram (EEG) activity during musical stimulation can be used to assess the consciousness states of patients with disorders of consciousness (DOC). In this study, a musical stimulation paradigm and verifiable criteria were used for c...

Despite previous efforts in depression detection studies, there is a scarcity of research on automatic depression detection using sleep structure, and several challenges remain: 1) how to apply sleep staging to detect depression and distinguish easily misjudged classes and 2) how to adaptively capture attentive channel-dimensional information to en...

Disorders of consciousness (DOCs) are often related to serious changes in sleep structure. This paper presents a sleep evaluation algorithm that scores the sleep structure of DOC patients to assist in assessing their consciousness level. The sleep evaluation algorithm is divided into two parts: 1) automatic sleep staging model: convolutional neural...

Few-shot learning (FSL) poses a considerable challenge since it aims to improve the model generalization ability with limited labeled data. Previous works usually attempt to construct class-specific prototypes and then predict novel classes using these prototypes. However, the feature distribution represented by the limited labeled data is coarse-g...

Assessing communication abilities in patients with disorders of consciousness (DOCs) is challenging due to limitations in the behavioral scale. Electroencephalogram-based brain-computer interfaces (BCIs) and eye-tracking for detecting ocular changes can capture mental activities without requiring physical behaviors and thus may be a solution. This...

Background Sleep spindles have emerged as valuable biomarkers for assessing cognitive abilities and related disorders, underscoring the importance of their detection in clinical research. However, template matching-based algorithms using fixed templates may not be able to fully adapt to spindles of different durations. Moreover, inspired by the mul...

In this work, a novel theoretical model of the void length probability distribution in 3D printed concrete is established based on a zigzag analog of the layer interface. A quasi-exponential distribution of void length is predicted and subsequently validated on both the zigzag analog and the actual 3D printed concrete, with different void ratios th...

Objective This study aimed to determine whether patients with disorders of consciousness (DoC) could experience neural entrainment to individualized music, which explored the cross-modal influences of music on patients with DoC through phase-amplitude coupling (PAC). Furthermore, the study assessed the efficacy of individualized music or preferred...

Attention decoding plays a vital role in daily life, where electroencephalography (EEG) has been widely involved. However, training a universally effective model for everyone is impractical due to substantial interindividual variability in EEG signals. To tackle the above challenge, we propose an end-to-end brain-computer interface (BCI) framework,...

Volume visualization plays a crucial role in both academia and industry, as volumetric data is extensively utilized in fields such as medicine, geosciences, and engineering. Addressing the complexities of volume rendering, neural rendering has emerged as a potential solution, facilitating the production of high-quality volume rendered images. In th...

Currently, traffic sign recognition techniques have been brought into the assistive driving of automobiles. However, small traffic sign recognition in real scenes is still a challenging task due to the class imbalance issue and the size limit of the traffic signs. To address the above issues, a feature‐enhanced hybrid attention network is proposed...

Dental caries has been widely recognized as one of the most prevalent chronic diseases in the field of public health. Despite advancements in automated diagnosis across various medical domains, it remains a substantial challenge for dental caries detection due to its inherent variability and intricacies. To bridge this gap, we release a hospital-sc...

Gait is one of the most popular biometrics for identity authentication today due to its noninvasive perception. Diverse spatial representations and temporal modeling are crucial information for gait recognition, especially under covariation conditions. However, most existing algorithms only focus on the specific temporal-scale modeling (i.e., short...

Objective and accurate detection of Parkinson’s disease (PD) is crucial for timely intervention and treatment. Electroencephalography (EEG) has been proven to characterize PD by measuring brain activity. In recent years, deep learning methods have gained great attention in automated PD detection, but their performance is limited by insufficient dat...

Electroencephalograms (EEGs) have garnered immense attention due to their security features, which are difficult to physically counterfeit in the field of person identification. Despite significant achievements in EEG-based person identification, several challenges remain: 1) how to dynamically update the model to identify an increasing number of u...

Accurate segmentation of anatomical structures or pathological lesions from medical images is crucial for reliable disease diagnosis and organ morphometry assessments in clinical practice. However, the presence of artifacts, occlusions, and uneven brightness introduced during image acquisition can present significant complexity in achieving segment...

Assessing consciousness in patients with disorders of consciousness (DOC) is frequent and crucial in clinical examination. However, current mainstream methods based on behavioral scales are time-consuming and prone to high misdiagnosis rates. Facial expressions holding crucial cues related to consciousness may offer a convenient and objective means...

Identifying the brain responses of patients with disorders of consciousness (DOCs), which include comas, vegetative states (VSs, also called unresponsive wakefulness syndrome (UWS)) and minimally conscious states (MCSs), based on electroencephalography (EEG) has important clinical diagnosis implications. However, due to impaired motor and cognitive...

Electroencephalography (EEG) is a commonly used technology for monitoring brain activities and diagnosing sleep disorders. Clinically, doctors need to manually stage sleep based on EEG signals, which is a time-consuming and laborious task. In this study, we propose a few-shot EEG sleep staging termed transductive prototype optimization network (TPO...

In this paper, a novel spatio-temporal self-constructing graph neural network (ST-SCGNN) is proposed for cross-subject emotion recognition and consciousness detection. For spatio-temporal feature generation, activation and connection pattern features are first extracted and then combined to leverage their complementary emotion-related information....

Background In recent years, road traffic safety has become a prominent issue due to the worldwide proliferation of vehicles on roads. The challenge of driver fatigue detection involves balancing the efficiency and accuracy of the detection process. While various detection methods are available, electroencephalography (EEG) is considered the gold st...

Deep learning-based AI diagnostic models rely heavily on high-quality exhaustive-annotated data for algorithm training but suffer from noisy label information. To enhance the model’s robustness and prevent noisy label memorization, this paper proposes a robust Semi-supervised Contrastive Learning paradigm called SSCL, which can efficiently merge se...

Sleep staging is significant for the capture of sleep patterns and the assessment of sleep quality. Although previous studies attempted to automatically detect sleep stages and achieved high classification performance, several challenges remain: 1) How to correctly classify the sleep stages end-to-end. 2) How to capture the representations and slee...

Major depressive disorder (MDD) is a common and serious mental health problem that has received increasing attention from both researchers and clinicians. Electroencephalography (EEG)-based automatic diagnosis of MDD has been explored in previous studies, but feature extraction remains an area for improvement. We propose a novel deep learning appro...

Sleep spindles are an electroencephalogram (EEG) biomarker of non-rapid eye movement (NREM) sleep and have important implications for clinical diagnosis and prognosis. However, it is challenging to accurately detect sleep spindles due to the complexity of the human brain and the uncertainty of neural mechanisms. To improve the reliability and objec...

Introduction Attention is a complex cognitive function of human brain that plays a vital role in our daily lives. Electroencephalogram (EEG) is used to measure and analyze attention due to its high temporal resolution. Although several attention recognition brain-computer interfaces (BCIs) have been proposed, there is a scarcity of studies with a s...

Recently, video-based micro-expression recognition (MER) applications have attracted attention in various scenarios. However, current deep learning-based MER methods frequently struggle with several challenges, such as insufficient data, difficulty in capturing subtle facial motions, and keyframe recognition. In this paper, we propose a robust MER...

Electroencephalogram (EEG) is a crucial and widely utilized technique in neuroscience research. In this paper, we introduce a novel graph neural network called the spatial-temporal graph attention network with a transformer encoder (STGATE) to learn graph representations of emotion EEG signals and improve emotion recognition performance. In STGATE,...

Thanks to the development of depth sensors and pose estimation algorithms, skeleton-based action recognition has become prevalent in the computer vision community. Most of the existing works are based on spatio-temporal graph convolutional network frameworks, which learn and treat all spatial or temporal features equally, ignoring the interaction w...

The brain-computer interface (BCI) provides a direct communication pathway between the human brain and external devices. However, the models trained for existing subjects perform poorly on new subjects, which is termed the subject calibration problem. In this paper, we propose a semi-supervised meta learning (SSML) method for subject-transfer calib...

Using task-dependent neuroimaging techniques, recent studies discovered a fraction of patients with disorders of consciousness (DOC) who had no command-following behaviors but showed a clear sign of awareness as healthy controls, which was defined as cognitive motor dissociation (CMD). However, existing task-dependent approaches might fail when CMD...

italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Goal : As an essential human-machine interactive task, emotion recognition has become an emerging area over the decades. Although previous attempts to classify emotions have achieved high performance, several challenges remain open: 1) How to effective...

A brain-computer interface (BCI) measures and analyzes brain activity and converts it into computer commands to control external devices. Traditional BCIs usually require full calibration, which is time-consuming and makes BCI systems inconvenient to use. In this study, we propose an online P300 BCI spelling system with zero or shortened calibratio...

Driver vigilance estimation is essential for fatigue and traffic accident reduction. Although previous algorithms for driver vigilance estimation have achieved high evaluation performance, several challenges remain open: 1) How to effectively utilize multimodal electroencephalography (EEG) and electrooculogram (EOG) signals remains challenging. 2)...

Recently, gait‐based age and gender recognition have attracted considerable attention in the fields of advertisement marketing and surveillance retrieval due to the unique advantage that gaits can be perceived at a long distance. Intuitively, age and gender can be recognised by observing people's static shape (e.g. different hairstyles between male...

A brain-computer interface (BCI) is a non-muscular communication technology that provides an information exchange channel for our brains and external devices. During the decades, BCI has made noticeable progress and has been applied in many fields. One of the most traditional BCI applications is the BCI speller. This article primarily discusses the...

Brain-computer interface (BCI) systems are often used to convert signals from brain activities into control commands through external devices. There are few studies on controlling a car by multimodality due to its difficulty in the current research. This paper proposes a hybrid BCI control system based on electroencephalography (EEG), electrooculog...

For patients with disorders of consciousness, such as unresponsive wakefulness syndrome (UWS) patients and minimally conscious state (MCS) patients, their long treatment cycle and high cost commonly put a heavy burden on the patient’s family and society. Therefore, it is vital to accurately diagnose and predict consciousness recovery for such patie...

In this study, a hybrid brain-computer interface (BCI) system combining P300 potential and emotion patterns was proposed to improve the performance of awareness detection. Two video clips were flashed randomly to evoke the P300 potential, while a laughing or crying video clip was used to induce the corresponding emotion pattern. The subjects were a...

Background: Using task-dependent neuroimaging techniques, recent studies discovered a fraction of patients with disorders of consciousness (DOC) who had no command-following behaviors but showed a clear sign of awareness, which was defined as cognitive motor dissociation (CMD). Although many efforts were made to identify the CMD, existing task-depe...

In recent years, researchers have made significant contributions to 3D face reconstruction with the rapid development of deep learning. However, learning-based methods often suffer from time and memory consumption. Simply removing network layers hardly solves the problem. In this study, we propose a solution that achieves fast and robust 3D face re...

Brain-computer interface (BCI) provides a direct communication pathway between human brain and external devices. Before a new subject could use BCI, a calibration procedure is usually required. Because the inter- and intra-subject variances are so large that the models trained by the existing subjects perform poorly on new subjects. Therefore, effe...

Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have potentially complementary characteristics that reflect the electrical and hemodynamic characteristics of neural responses, so EEG-fNIRS-based hybrid brain-computer interface (BCI) is the research hotspots in recent years. However, current studies lack a comprehensiv...

Parkinson’s disease (PD) is a complex neurodegenerative disease. At present, the early diagnosis of PD is still extremely challenging, and there is still a lack of consensus on the brain characterization of PD, and a more efficient and robust PD detection method is urgently needed. In order to further explore the features of PD based on brain activ...

In recent years, neuroimaging studies have remarkably demonstrated the presence of cognitive motor dissociation in patients with disorders of consciousness (DoC). These findings accelerated the development of brain–computer interfaces (BCIs) as clinical tools for behaviorally unresponsive patients. This article reviews the recent progress of BCIs i...

Dynamic facial expression recognition (DFER) is a promising research area because it concerns the dynamic change pattern of facial expressions, but it is difficult to effectively capture the facial appearances and dynamic temporal information of each image in an image sequence. In this paper, a cascaded spatiotemporal attention network (CSTAN) is p...

Behavioral assessment of sound localization in the Coma Recovery Scale-Revised (CRS-R) poses a significant challenge due to motor disability in patients with disorders of consciousness (DOC). Brain-computer interfaces (BCIs), which can directly detect brain activities related to external stimuli, may thus provide an approach to assess DOC patients...

This study proposes a brain-computer interface (BCI)- and Internet of Things (IoT)-based smart ward collaborative system using hybrid signals. The system is divided into hybrid asynchronous electroencephalography (EEG)-, electrooculography (EOG)- and gyro-based BCI control system and an IoT monitoring and management system. The hybrid BCI control s...

At present, emotion recognition based on electroencephalograms (EEGs) has attracted much more attention. Current studies of affective brain-computer interfaces (BCIs) focus on the recognition of happiness and sadness using brain activation patterns. Fear recognition involving brain activities in different spatial distributions and different brain f...

The human-machine interface (HMI) has been studied for robot teleoperation with the aim of empowering people who experience motor disabilities to increase their interaction with the physical environment. The challenge of an HMI for robot control is to rapidly, accurately, and sufficiently produce control commands. In this paper, an asynchronous HMI...

Music can effectively improve people's emotions, and has now become an effective auxiliary treatment method in modern medicine. With the rapid development of neuroimaging, the relationship between music and brain function has attracted much attention. In this study, we proposed an integrated framework of multi-modal electroencephalogram (EEG) and f...