Xiaomeng Su’s research while affiliated with Norwegian University of Science and Technology and other places

This study explores the synchronization of multimodal physiological data streams, in particular, the integration of electroencephalography (EEG) with a virtual reality (VR) headset featuring eye-tracking capabilities. A potential use case for the synchronized data streams is demonstrated by implementing a hybrid steady-state visually evoked potential (SSVEP) based brain-computer interface (BCI) speller within a fully immersive VR environment. The hardware latency analysis reveals an average offset of 36 ms between EEG and eye-tracking data streams and a mean jitter of 5.76 ms. The study further presents a proof of concept brain-computer interface (BCI) speller in VR, showcasing its potential for real-world applications. The findings highlight the feasibility of combining commercial EEG and VR technologies for neuroscientific research and open new avenues for studying brain activity in ecologically valid VR environments. Future research could focus on refining the synchronization methods and exploring applications in various contexts, such as learning and social interactions.

Background Technological advancements in functional neuroimaging and motion capture have led to the development of novel methods that facilitate the diagnosis and rehabilitation of motor deficits. These advancements allow for the synchronous acquisition and analysis of complex signal streams of neurophysiological data (e.g., EEG, fNIRS) and behavioral data (e.g., motion capture). The fusion of those data streams has the potential to provide new insights into cortical mechanisms during movement, guide the development of rehabilitation practices, and become a tool for assessment and therapy in neurorehabilitation. Research objective This paper aims to review the existing literature on the combined use of motion capture and functional neuroimaging in motor rehabilitation. The objective is to understand the diversity and maturity of technological solutions employed and explore the clinical advantages of this multimodal approach. Methods This paper reviews literature related to the combined use of functional neuroimaging and motion capture for motor rehabilitation following the PRISMA guidelines. Besides study and participant characteristics, technological aspects of the used systems, signal processing methods, and the nature of multimodal feature synchronization and fusion were extracted. Results Out of 908 publications, 19 were included in the final review. Basic or translation studies were mainly represented and based predominantly on healthy participants or stroke patients. EEG and mechanical motion capture technologies were most used for biomechanical data acquisition, and their subsequent processing is based mainly on traditional methods. The system synchronization techniques at large were underreported. The fusion of multimodal features mainly supported the identification of movement-related cortical activity, and statistical methods were occasionally employed to examine cortico-kinematic relationships. Conclusion The fusion of motion capture and functional neuroimaging might offer advantages for motor rehabilitation in the future. Besides facilitating the assessment of cognitive processes in real-world settings, it could also improve rehabilitative devices’ usability in clinical environments. Further, by better understanding cortico-peripheral coupling, new neuro-rehabilitation methods can be developed, such as personalized proprioceptive training. However, further research is needed to advance our knowledge of cortical-peripheral coupling, evaluate the validity and reliability of multimodal parameters, and enhance user-friendly technologies for clinical adaptation.

Example of data for using the code (git@github.com:BiomedicalEngineeringTechies/CLET.git) described in this article: CLET: Computation of Latencies in Event-related potential Triggers using photodiode on virtual reality apparatuses ( https://www.frontiersin.org/articles/10.3389/fnhum.2023.1223774/full). CLET is a semi-automated method to accurately compute the latencies (i.e., time delay) between triggers (or markers) used in EEG/ERP experiments.

To investigate event-related activity in human brain dynamics as measured with EEG, triggers must be incorporated to indicate the onset of events in the experimental protocol. Such triggers allow for the extraction of ERP, i.e., systematic electrophysiological responses to internal or external stimuli that must be extracted from the ongoing oscillatory activity by averaging several trials containing similar events. Due to the technical setup with separate hardware sending and recording triggers, the recorded data commonly involves latency differences between the transmitted and received triggers. The computation of these latencies is critical for shifting the epochs with respect to the triggers sent. Otherwise, timing differences can lead to a misinterpretation of the resulting ERPs. This study presents a methodical approach for the CLET using a photodiode on a non-immersive VR (i.e., LED screen) and an immersive VR (i.e., HMD). Two sets of algorithms are proposed to analyze the photodiode data. The experiment designed for this study involved the synchronization of EEG, EMG, PPG, photodiode sensors, and ten 3D MoCap cameras with a VR presentation platform (Unity). The average latency computed for LED screen data for a set of white and black stimuli was 121.98 ± 8.71 ms and 121.66 ± 8.80 ms, respectively. In contrast, the average latency computed for HMD data for the white and black stimuli sets was 82.80 ± 7.63 ms and 69.82 ± 5.52 ms. The codes for CLET and analysis, along with datasets, tables, and a tutorial video for using the codes, have been made publicly available.

Business Intelligence and Analytics (BIA) systems play an essential role in organizations, providing actionable insights that enable business users to make more informed, data-driven decisions. However, many Higher Education (HE) institutions do not have accessible and usable models to guide them through the incremental development of BIA solutions to realize the full potential value of BIA. The situation is becoming ever more acute as HE operates today in a complex and dynamic environment brought forward by globalization and the rapid development of information technologies. This paper proposes a domain-specific BIA maturity model (MM) for HE–the HE-BIA Maturity Model. Following a design science approach, this paper details the design, development, and evaluation of two artifacts: the MM and the maturity assessment method. The evaluation phase comprised three case studies with universities from different countries and two workshops with practitioners from more than ten countries. HE institutions reported that the assessment with the HE-BIA model was (i) useful and adequate for their needs; (ii) and contributed to a better understanding of the current status of their BIA landscape, making it explicit that a BIA program is a technology endeavor as well as an organizational development.

Background Balance training exercise games (exergames) are a promising tool for reducing fall risk in elderly. Exergames can be used for in-home guided exercise, which greatly increases availability and facilitates independence. Providing biofeedback on weight-shifting during in-home balance exercise improves exercise efficiency, but suitable equipment for measuring weight-shifting is lacking. Exergames often use kinematic data as input for game control. Being able to useg such data to estimate weight-shifting would be a great advantage. Machine learning (ML) models have been shown to perform well in weight-shifting estimation in other settings. Therefore, the aim of this study was to investigate the performance of ML models in estimation of weight-shifting during exergaming using kinematic data. Methods Twelve healthy older adults (mean age 72 (± 4.2), 10 F) played a custom exergame that required repeated weight-shifts. Full-body 3D motion capture (3DMoCap) data and standard 2D digital video (2D-DV) was recorded. Weight shifting was directly measured by 3D ground reaction forces (GRF) from force plates, and estimated using a linear regression model, a long-short term memory (LSTM) model and a decision tree model (XGBoost). Performance was evaluated using coefficient of determination ( $R^2$ R 2 ) and root mean square error (RMSE). Results Results from estimation of GRF components using 3DMoCap data show a mean (± 1SD) RMSE (% total body weight, BW) of the vertical GRF component ( $F_z$ F z ) of 4.3 (2.5), 11.1 (4.5), and 11.0 (4.7) for LSTM, XGBoost and LinReg, respectively. Using 2D-DV data, LSTM and XGBoost achieve mean RMSE (± 1SD) in $F_z$ F z estimation of 10.7 (9.0) %BW and 19.8 (6.4) %BW, respectively. $R^2$ R 2 was $>.97$ > . 97 for the LSTM in the $F_z$ F z component using 3DMoCap data, and $>.77$ > . 77 using 2D-DV data. For XGBoost, $F_z$ F z $R^2$ R 2 was $>.86$ > . 86 using 3DMoCap data, and $>.56$ > . 56 using 2D-DV data. Conclusion This study demonstrates that an LSTM model can estimate 3-dimensional GRF components using 2D kinematic data extracted from standard 2D digital video cameras. The $F_z$ F z component is estimated more accurately than $F_y$ F y and $F_x$ F x components, especially when using 2D-DV data. Weight-shifting performance during exergaming can thus be extracted using kinematic data only, which can enable effective independent in-home balance exergaming.

Use of VR-games is considered a promising treatment approach in stroke rehabilitation. However, there is little knowledge on the use and expectations of patients and health professionals regarding the use of treadmill walking in a fully immersive virtual environment as a rehabilitation tool for gait training for stroke survivors. The objectives of the current study were to determine whether stroke survivors can use fully immersive VR utilizing modern HMDs while walking on a treadmill without adverse effects, and to investigate the experiences of stroke survivors and clinicians after testing with focus on acceptability and potential utilization in rehabilitation. A qualitative research design with semi-structured interviews was used to collect data. Five stroke survivors and five clinicians participated in the study and tested a custom-made VR-game on the treadmill before participating in individual semi-structured interview. Data were analyzed through thematic analysis. The analysis of the interview data identified two main categories: (1) experiencing acceptability through safety and motivation, and (2) implementing fully immersive VR in rehabilitation. Both stroke survivors' and clinicians enjoyed the treadmill-based VR-game and felt safe when using it. The stroke survivors experienced motivation for exercising and achievement by fulfilling tasks during the gaming session as the VR-game was engaging. The clinicians found additional motivation by competing in the game. Both groups saw a potential for use in gait rehabilitation after stroke, on the premise of individual adaptation to each patient's needs, and the technology being easy to use. The findings from this qualitative study suggest that a fully immersive treadmill-based VR-game is acceptable and potentially useful as part of gait rehabilitation after stroke, as it was positively received by both stroke survivors and clinicians working within stroke rehabilitation. The participants reported that they experienced motivation in the game through safety, engagement and achievement. They also saw the potential of implementing such a setup in their own rehabilitation setting. Elements that enable safety and engaging experience are important to maintain when using a fully immersive VR-game in stroke rehabilitation.

In exercise gaming (exergaming), reward systems are typically based on rules/templates from joint movement patterns. These rules or templates need broad ranges in definitions of correct movement patterns to accommodate varying body shapes and sizes. This can lead to inaccurate rewards and, thus, inefficient exercise, which can be detrimental to progress. If exergames are to be used in serious settings like rehabilitation, accurate rewards for correctly performed movements are crucial. This article aims to investigate the level of accuracy machine learning/deep learning models can achieve in classification of correct repetitions naturally elicited from a weight-shifting exergame. Twelve healthy elderly (10F, age 70.4 SD 11.4) are recruited. Movements are captured using a marker-based 3-D motion-capture system. Random forest (RF), support vector machine, k-nearest neighbors, and multilayer perceptron (MLP) are the employed models, trained and tested on whole body movement patterns and on subsets of joints. MLP and RF reached the highest recall and F1-score, respectively, when using combined data from joint subsets. MLP recall range are 91% to 94%, and RF F1-score range 79% to 80%. MLP and RF also reached the highest recall and F1-score in each joint subset, respectively. Here, MLP ranged from 93% to 97% recall, while RF ranged from 73% to 80% F1-score. Recall results, show that >9 out of 10 repetitions are classified correctly, indicating that MLP/RF can be used to identify correctly performed repetitions of a weight-shifting exercise when using full-body data and when using joint subset data.

Maturity models of Business Intelligence and Analytics (BIA) have been previously used to assess BIA development progress in organizations in many sectors, such as healthcare and business. However, there is a lack of studies reporting up-to-date knowledge on applying maturity assessment in Higher Education Institutions (HEI). It remains unclear precisely to what extent and how HEI employ maturity assessment and the benefits of such exercises. This paper addresses this gap by reporting a case study at a large Norwegian university. A domain-specific maturity model is used as a lens to observe and reflect on the BIA implementation at the Norwegian University of Science and Technology. This paper reports the assessment results and discuss the implications of the maturity assessment. The findings and discussions in the case can cater to a broader audience of BIA practitioners and researchers, contributing to understanding the value and adoption dynamics of BIA in Higher Education.