Honghai Liu

Honghai Liu
University of Portsmouth

PhD at King's College London

About

424
Publications
88,221
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
9,736
Citations

Publications

Publications (424)
Preprint
Full-text available
Skeleton-based Temporal Action Segmentation involves the dense action classification of variable-length skeleton sequences. Current approaches primarily apply graph-based networks to extract framewise, whole-body-level motion representations, and use one-hot encoded labels for model optimization. However, whole-body motion representations do not ca...
Article
3D Multi-Object Tracking (MOT) aims to establish and maintain consistent object trajectories in continuously dynamic environments. At present, the tracking-by-detection has emerged as a dominant paradigm for 3D MOT, due to its simplicity and efficiency. However, this paradigm depends heavily on the performance of 3D object detection, which usually...
Article
Neurorobotic devices, such as prostheses, exoskeletons, and muscle stimulators, can partly restore motor functions in individuals with disabilities, such as stroke, spinal cord injury (SCI), and amputations and musculoskeletal impairments. These devices require information transfer from and to the nervous system by neurorobotic interfaces. However,...
Article
Noisy labels make a profoundly negative impact on Facial Expression Recognition (FER) due to inter-class similarity and subjective annotation. Recent works mainly focus on distinguishing the clean samples or mining the latent truth, which not only need extra computational overhead, but also have the error risk of relabeling. In this work, we propos...
Preprint
We have identified a potential method for unifying first-order optimizers through the use of variable Second-Moment Exponential Scaling(SMES). We begin with back propagation, addressing classic phenomena such as gradient vanishing and explosion, as well as issues related to dataset sparsity, and introduce the theory of balance in optimization. Thro...
Article
Interpersonal communication facilitates symptom measures of autistic sociability to enhance clinical decision-making in identifying children with autism spectrum disorder (ASD). Traditional methods are carried out by clinical practitioners with assessment scales, which are subjective to quantify. Recent studies employ engineering technologies to an...
Article
Full-text available
In certain neurological disorders such as stroke, the impairment of upper limb function significantly impacts daily life quality and necessitates enhanced neurological control. This poses a formidable challenge in the realm of rehabilitation due to its intricate nature. Moreover, the plasticity of muscle synergy proves advantageous in assessing the...
Article
Human-Object Interaction (HOI) detection plays a vital role in scene understanding, which aims to predict the HOI triplet in the form of . Existing methods mainly extract multi-modal features (e.g., appearance, object semantics, human pose) and then fuse them together to directly predict HOI triplets. However, most of these methods focus on seeking...
Article
Human-Object Interaction (HOI) detection aims to infer interactions between humans and objects, and it is very important for scene analysis and understanding. The existing methods usually focus on exploring instance-level (e.g., object appearance) or interaction-level (e.g., action semantic) features to conduct interaction prediction. However, most...
Article
Real-world Facial Expression Recognition (FER) suffers from noisy labels due to ambiguous expressions and subjective annotation. Overall, addressing noisy label FER involves two core issues: the efficient utilization of clean samples and the effective utilization of noisy samples. However, existing methods demonstrate their effectiveness solely thr...
Article
Facial Expression Recognition (FER) aims to categorize emotional expressions depicted on a human face, and is a challenging task under unconstrained conditions, such as face occlusions and pose variations. Recent methods usually adopt self attention or cross attention to explore global or local relationships among different level features. However,...
Article
In the fields of health care and clinical monitoring, vision-based Facial Expression Recognition (FER) has achieved significantly progress, but it still faces the challenge of poor generalization ability under unconstrained conditions of occlusions and pose variation. Recently, Vision-Language Model (VLM) has greatly advanced the FER task. However,...
Article
Corticomuscular coupling (CMC) refers to the interaction between neural activity in the cerebral cortex (especially the motor cortex) and muscle activity, is an effective method for investigating the neural mechanisms of motor control. Achieving robust CMC analysis depends significantly on the fidelity of electroencephalogram (EEG) and electromyogr...
Article
Full-text available
Understanding electrotactile parametric properties is a crucial milestone in achieving intuitive haptics. Perceptual intensity is a primary property, but its exploration remains challenging due to subjectivity. To address this problem, this study conducted two experiments on fingertips and proposed two metrics based on significant findings. Experim...
Conference Paper
The interaction of information between the brain and muscles is widely present in human activities. Brain-muscle coupling has been proven to quantify the amount of information transmission between the cortex and muscles. Motor dysfunction is a typical sequela of stroke. Accurate quantitative assessment of motor function is the basis for formulating...
Conference Paper
Given an untrimmed video, repetitive actions counting aims to estimate the number of repetitions of class-agnostic actions. To handle the various length of videos and repetitive actions, also optimization challenges in end-to-end video model training, down-sampling is commonly utilized in recent state-of-the-art methods, leading to ignorance of sev...
Conference Paper
To address the issue of the unclear factors affecting the perception performance of biomimetic tactile sensing system, FEM and cantilever beam model are firstly employed to analyze the effects of the physical properties of biomimetic multi-layer elastomer on the deformation and vibration tactile signals. Next, the impacts of physical properties and...
Article
The sensitivity of human tactile perception is essential in sensing texture surfaces via the fingertip. However, direct investigation of the underlying physiological principles is challenging due to their complexity, while bionic vision-based tactile sensing is a promising solution. This paper proposed a novel high-speed vision-based tactile sensor...
Article
Hand gesture recognition plays a crucial role in the field of human–computer interaction (HCI). In terms of the multimodal sensing of hand gestures, the A-mode ultrasound (AUS) signal is far less investigated, especially for dynamic hand gestures, than its counterparts, such as surface electromyography (sEMG). In this article, we explore the recogn...
Preprint
We propose the simplest SGD enhanced method ever, Loss-Controlled Asymmetric Momentum(LCAM), aimed directly at the Saddle Point problem. Compared to the traditional SGD with Momentum, there's no increase in computational demand, yet it outperforms all current optimizers. We use the concepts of weight conjugation and traction effect to explain this...
Article
This article provides a comprehensive review of the current state of noninvasive brain imaging and brain stimulation in motor rehabilitation after stroke. The functional organization of the brain is determined by brain network connections. Brain imaging enables us to map the network organization of the brain, so as to assess the motor function in a...
Article
Objective: Autism Spectrum Disorders (ASD) are characterized by impairments in joint attention (JA) comprising two components: responding to JA (RJA) and initiating JA (IJA). RJA and IJA are considered two interrelated aspects of JA, related to different stages of infant development. While recent technologies have been used to characterize RJA eme...
Article
Objective: Post-stroke transcranial magnetic stimulation (TMS) has gradually become a brain intervention to assist patients in the recovery of motor function. The long lasting regulatory of TMS may involve the coupling changes between cortex and muscles. However, the effects of multi-day TMS on motor recovery after stroke is unclear. Methods: Th...
Article
Full-text available
Copious evidence shows that impaired visual–motor integration (VMI) is intrinsically linked to the core deficits of autism spectrum disorder (ASD) and associated with an anomalous social capability. Therefore, an effective evaluation method of visual–motor behaviour can provide meaningful insight into the evaluation of VMI towards social capability...
Article
Gaze is a vital feature in analyzing natural human behavior and social interaction. Existing gaze target detection studies learn gaze from gaze orientations and scene cues via a neural network to model gaze in unconstrained scenes. Though achieve decent accuracy, these studies either employ complex model architectures or leverage additional depth i...
Article
3-D Morphable model (3DMM) has widely benefited 3-D face-involved challenges given its parametric facial geometry and appearance representation. However, previous 3-D face reconstruction methods suffer from limited power in facial expression representation due to the unbalanced training data distribution and insufficient ground-truth 3-D shapes. In...
Article
It is challenging to have a unified roughness metric representing the tactile sensation given the complexity of surface properties like hardness, viscosity, and texture. In this paper, a metric is proposed to evaluate tactile roughness across materials based on the granularity commonly used for sandpapers and implemented through a combination of se...
Article
It is evident that the state of the art in multi-sensory hand gesture recognition indicates the superior performance of A-mode ultrasound (AUS) modality over its counterparts. Nevertheless, the practical implementation of AUS-based gesture recognition in real-life scenarios is impeded by its susceptibility to muscle fatigue, thereby limiting its lo...
Article
Facial Expression Recognition (FER) aims to identify emotional expressions in human faces, and it is a fundamental task in computer vision. Recently, some methods apply Vision Transformer (ViT) to FER and have achieved promising results. However, FER still suffers from two key issues: inter-class similarity and intra-class discrepancy. To address t...
Preprint
The recent trend in multiple object tracking (MOT) is jointly solving detection and tracking, where object detection and appearance feature (or motion) are learned simultaneously. Despite competitive performance, in crowded scenes, joint detection and tracking usually fail to find accurate object associations due to missed or false detections. In t...
Article
Full-text available
Due to low energy consumption and fast response, electrotactile feedback has shown great potential in human-machine interaction. However, regulating electrotactile perception remains challenging because of the high variability of electrode-skin impedance. Electrode-skin modelling is a common solution, but current researches are still facing many pr...
Article
Surface electromyogram (sEMG)-based hand gesture recognition, which interprets commands given by humans through sEMG signals, performs well in many studies. However, its recognition accuracy drops dramatically due to electrode shift since the distributions of motion classes are changed. Although calibrating the system with newly collected samples a...
Article
Full-text available
Ultrasound can non-invasively detect muscle deformations and has great potential applications in prosthetic hand control. Traditional ultrasound equipment was usually too bulky to be applied in wearable scenarios. This research presented a compact ultrasound device that could be integrated into a prosthetic hand socket. The miniaturized ultrasound...
Article
Full-text available
Simultaneous prediction of wrist and hand motions is essential for the natural interaction with hand prostheses. In this paper, we propose a novel multi-out Gaussian process (MOGP) model and a multi-task deep learning (MTDL) algorithm to achieve simultaneous prediction of wrist rotation (pronation/ supination)1 and finger gestures for transradial a...
Chapter
Joint attention is an important skill that involves coordinating the attention of at least two individuals towards an object or event in early child development, which is usually absent in children with autism. Children’s joint attention is an essential part of the diagnosis of autistic children. To improve the effectiveness of autism screening, in...
Chapter
Neurodevelopmental disorder refers to behavioral and cognitive impairment during development, which is manifested as significant difficulties in intelligence, motor or social skills. Among these disorders, autism spectrum disorder (ASD), language disorder (LD) and mental retardation (MR) are easily misclassified due to their similar symptoms, which...
Chapter
Autism spectrum disorders is a range of neurodevelopmental conditions primarily characterized by difficulties in social interactions, differences in communication, and presentations of rigid and repetitive behavior. The evidence shows that the functional social behavior of children with autism can be enhanced by early intervention. However, traditi...
Chapter
Human visual focus is a vital feature to uncover subjects’ underlying cognitive processes. To predict the subject’s visual focus, existing deep learning methods learn to combine the head orientation, location, and scene content for estimating the visual focal point. However, these methods mainly face three problems: the visual focal point predictio...
Chapter
Fatigue monitoring is significant during movement process to avoid body injury cased by excessive exercise. To address this issue, we developed an automated framework to recognize human fatigue states based on electrocardiogram (ECG) collected by a smart wearable device. After preprocessing on the raw ECG data, both machine learning solution and de...
Chapter
Autism Spectrum Disorder (ASD) is a congenital neurodevelopmental disorder, and the number of ASD has been increasing in recent decades worldwide. Early screening is essential for proper treatment and intervention in toddlers with ASD. However, manual early screening methods for ASD are costly and inefficient. Stereotyped behavior is one of the cli...
Chapter
Full-text available
To sense and represent electrotactile perceptual properties is a crucial milestone in order to achieve intuitive haptics. However, electrotactile perceptual properties are very poorly studied. This study presented an experimental study on the electrotactile perceptual properties of fingertips. A series of experimental paradigms were designed based...
Chapter
Stroke is a common global health-care problem that has had serious negative impact on the life quality of the patients. Motor impairment after stroke typically affects one side of the body. The main focus of stroke rehabilitation is the recovery of the affected neuromuscular functions and the achievement of independent body control. This paper prop...
Chapter
Robots with tactile sensors can distinguish the tactile property of the object, such as the spatial shape, in many robotic applications. The neuromorphic approach offers a new solution for information processing to encode tactile signals. Vision-based tactile sensing has gradually attracted attention in recent years. Although some work has been don...
Chapter
Stroke has been one of the diseases with high incidence in the world, which is the main reason for adult deformity. It’s significant to diagnose stroke quickly and accurately. Currently the main diagnostic method of acute ischemic stroke is also Computed Tomography. Meanwhile, electroencephalogram (EEG) is an electrophysiological manifestation that...
Chapter
IS (Interpersonal Synchrony), where the follower (participant) tries to behave the same action along with the raiser (human or metronome), is an essential social interaction skill. The evaluation of interpersonal synchronization is valuable for early autism screening. However, the research on IS evaluation is limited, and the current approaches usu...
Chapter
Face alignment has been an important focus of vision research because it is the most fundamental step in face analysis, reconstruction, and applications of emotion and attention. However, face alignment still suffers from some problems, such as lack of stability and poor performance in practical applications due to occlusion, illumination, and high...
Chapter
In the diagnosis and rehabilitation of motor function for stroke patients, the combination of motor function assessment based on Muscle Synergy Analysis (MSA) and rehabilitation using Functional Electrical Stimulation (FES) is a new strategy, which has been validated its feasibility and superiority in clinical rehabilitation. However, it is difficu...
Chapter
At present, due to the limited information, the single man-machine interface control has some defects in human motion prediction, such as low accuracy and poor robustness. In this work, a multi-modal real-time acquisition system that combines surface electromyography (sEMG), inertial measurement (IMU) and A-mode ultrasound (AUS) information is used...
Chapter
Ultrasound can non-invasively detect muscle deformations, which has great potential applications in prosthetic hand control. This research developed a miniaturized ultrasound device that could be integrated into a prosthetic hand socket. This compact system included four A-mode ultrasound transducers, a signal processing module, and a prosthetic ha...
Chapter
In recent years, gaze estimation has been applied to numerous application areas, such as driver monitor system, autism assessment, and so on. However, current practical gaze estimation algorithms require a large amount of data to obtain better results. The collection of gaze data requires specific equipment, and the collection process is cumbersome...
Chapter
To explore the potential of gesture recognition based on the A-mode ultrasound (AUS) interface in human-computer interaction (HCI), according to the characteristics of AUS signal, a novel preprocessing approach is designed, feature extraction is performed by the window analysis method, and four methods, Linear Discriminant Analysis (LDA), k-Nearest...
Chapter
The sEMG (surface electromyogram signal) reflects the degree of muscle activity, which contains a large amount of motion information about the human body. It has important research significance and application value in sports science, rehabilitation medicine, and so on. There are more and more products related to sEMG acquisition, which are mainly...
Chapter
Insufficient analgesia during resuscitation of postoperative children may lead to sequelae such as atelectasis and worsening respiratory function, so timely and accurate detection of pain status in children is essential. At present, the pain status of children is mainly judged by nurses based on facial pain expression scales, which is highly subjec...
Chapter
Autism is a childhood developmental disorder characterized by impairments in social interaction, verbal and nonverbal communication, narrow interests, and repetitive stereotypes, starting in infancy and early childhood. Children’s Response to Instruction (RTI) is an important protocol in the diagnosis of children with autism. The so-called verbal r...
Chapter
Multi-object tracking (MOT) is an important problem in computer vision which has a wide range of applications. Formulating MOT as multi-task learning of object detection and re-Identification (re-ID) in a single network is appealing since it achieves real-time but effective inference on detection and tracking. However, in crowd scenes, the existing...
Article
Full-text available
Isokinetic exercise is considered as one of the most effective ways for rehabilitation and muscle strength enhancement. As rehabilitation equipment, the isokinetic exercise device assists isokinetic movement using a single fixed axis and relevant joint adapters, and it requires high stability and safety performances to cooperate with human. In this...
Article
Full-text available
With the increased demands of human-machine interaction, haptic feedback is becoming increasingly critical. However, the high cost, large size and low efficiency of current haptic systems severely hinder further development. As a portable and efficient technology, cutaneous electrotactile stimulation has shown promising potential for these issues....
Article
Objective: Transcranial magnetic stimulation (TMS) is an experimental therapy for promoting motor recovery from hemiparesis. At present, hemiparesis patients' responses to TMS are variable. To maximize its therapeutic potential, we need an approach that relates the electrophysiology of motor recovery and TMS. To this end, we propose Corticomuscula...
Article
Influenced by multiple factors, to understand electrotactile properties is challenging but significant in comparing individual differences, parameters control and coding, and model studies. However, the inadequacy of measurement systems and methods limits the application and development of electrotactile feedback. This study focuses on the explorat...
Article
Full-text available
Due to visual impairment and lack of social attention, the reading problem of the blind has not been well resolved. Many existing Braille reading devices still have many problems and are not accepted by the public. This paper developed a portable Braille reading system based on electrotactile display technology. The proposed system is composed of a...
Article
The muscle synergy hypothesis assumes that the nervous system controls muscles in groups to simplify behavioral tasks, which makes it possible for modularizing motor function assessment. This paper presents a new assessment method based on muscle synergy space (MSS) model to assess motor functions after stroke. It consists of spatiotemporal feature...
Article
Brain networks allow a topological understanding into the pathophysiology of stroke-induced motor deficits, and have been an influential tool for investigating brain functions. Unfortunately, currently applied methods generally lack in the recognition of the dynamic changes in the cortical networks related to muscle activity, which is crucial to cl...
Chapter
Adaptive behavioural assessments are useful in the diagnosis of autism. This research proposes a strategy of assessing autistic children’s adaption skills through the change of hand behaviour complexity based on deep learning and complex systems. Specifically, we implement a sparse representation of high-dimensional features of hand movements utili...
Chapter
In recent years, due to the defects of weak sEMG signal, insensitive to fine finger movement and serious impression by noise, researchers consider the need to use A-mode ultrasound (AUS) for gesture decoding. However, the current A-mode ultrasonic gesture recognition algorithm is still relatively basic, which can recognize the recognition function...
Article
Surface electromyography (sEMG) and A-mode ultrasound (AUS) are two widely employed sensing modalities to detect muscle activities. By comparison, the AUS modality shows the characteristics of higher decoding accuracy than the sEMG modality. However, AUS is far less reliable than sEMG in actual long-term use. To resolve this contradiction, we consi...
Article
Human hands play a very important role in daily object manipulation. Current prosthetic hands are capable of mimicking most functions of the human hand, but how to interact with prosthetic hands based on human intentions remains an open problem. In this article, we proposed a wearable ultrasound-based interface to achieve simultaneous and proportio...
Article
The hypothesis of muscle synergy has been dominantly accepted in research due to the evidence of being well investigated its existence in multiple disciplines such as health science. Motor function assessment using muscle synergy provides a quantitative approach from the physiological features of muscles. Given the synergy coordinating multiple mus...
Article
Though physiological signal based human-machine interfaces (HMIs) have recently developed rapidly, their practical use is restricted by many real-world environmental factors, one of which is muscle fatigue. This paper explores the sensitivities between surface electromyography (sEMG) and A-mode ultrasound (AUS) sensing modalities subject to muscle...
Article
Motion estimation plays an important role in motor coordination exploration and optimal control of human-machine interaction. Muscular strength training, such as isokinetic exercise, aims to improve one’s muscular strength and endurance with adjustable resistance at a constant speed. To explore motion estimation and muscle coordination, this paper...
Article
The acoustic nonlinearity parameter of skeletal muscles changes with their contractions and has the potential to be utilized for exoskeleton control. In our previous work, a system comprising a wearable ultrasound probe was developed to estimate the acoustic nonlinearity parameter in vivo by measuring amplitudes of second-harmonic ultrasound. The...
Chapter
It is evident that surface electromyography (sEMG) based prosthesis is constrained due to sensitivity to muscle fatigue. This paper investigated the muscle fatigue robustness for sEMG, ultrasound and the fusion sEMG/ultrasound signals towards the proportional force prediction. The linear regression model is developed, and evaluated on the non-fatig...
Chapter
Early screening of autism spectrum disorder is essential since early intervention can enhance the functional social behavior of autistic children. Among the behavior characteristics of autism, pointing is strongly related to human communicative interaction and cognitive development. To improve the efficiency and accuracy of autism screening, this p...
Chapter
This paper proposed learning from demonstration control policy that permits an industrial robotic arm to generalise in an unstructured and dynamic environment. The approach combines a probabilistic model with a reactive approach that learns the cost function of an unknown state of the environmental constraints. The approach redefines the robot’s be...
Article
Accurate force control plays a crucial role but still faces many challenges in functional electrical stimulation (FES) based neuroprosthesis applications. This study proposes an surface electromyography (sEMG) driven solution for FES sequential parameter estimation including its amplitude and pulsewidth (PW). The algorithm forms the mapping relatio...
Article
The existing slope safety monitoring studies mainly focus on long-term monitoring of slowly evolving horizontal deformations. However, during the practical applications, sudden deformations may also lead to landslides due to the natural and man-made trigger factors. This paper first proposes a reusable and distributed FBG inclinometer for simultane...
Article
Objective: While the corticomuscular coupling between motor cortex and muscle tissue has received considerable attention, which is typically quantitative measure to evaluate neural signals synchronization in the motor control system, little work has been published regarding the effect of underlying delay of two coupled physiological signals on coh...
Article
Full-text available
The greatest challenge in autism spectrum disorder (ASD) detection and treatment arises from the elusive etiological origins and heterogeneity of ASD. ET-EEG correlative analytics refers to correlating simultaneously recorded Eye tracking (ET) and Electroencephalography (EEG) data or combining them to yield diagnostic biomarkers. This approach was...