Tianyu Shen’s research while affiliated with Beijing University of Chemical Technology and other places

Intelligent control is a crucial technology for realizing Industry 5.0, which makes industrial engineering systems more efficient, robust, and resilient. It is noteworthy that uncertainties and disturbances will inevitably be detrimental to the control performances of Industry 5.0 engineering applications. To deal with these issues, we propose a novel super-twisting-like continuous extended state observer-based fixed-time composite learning fuzzy control scheme and apply it to a typical engineering system. Unlike conventional fixed-time adaptive fuzzy control methods that update parameters merely by closed-loop stability conditions, the proposed fixed-time control scheme utilizes both tracking errors and prediction errors to update parameters compositely, which achieves better-tracking performance and fuzzy approximation precision. First, fuzzy logic systems (FLSs) are developed to identify the unknown model functions in the Industry 5.0 engineering system. Second, to deal with the remaining approximation errors of the FLSs, parameter uncertainties, and external disturbances, the novel super-twisting-like continuous extended state observers are designed to estimate these lumped disturbances. Third, the prediction errors that indicate the fuzzy approximation precision are constructed by developing fixed-time parallel estimators. Moreover, rigorous Lyapunov stability analysis is carried out to illustrate the fixed-time convergence of the entire closed-loop control system. Finally, the proposed control scheme is applied to a practical buck converter engineering system toward Industry 5.0, and comparative hardware experiments verified the advantages of the control scheme.

The tremendous impact of large models represented by ChatGPT [1]–[3] makes it necessary to consider the practical applications of such models [4]. However, for an artificial intelligence (AI) to truly evolve, it needs to possess a physical “body” to transition from the virtual world to the real world and evolve through interaction with the real environments. In this context, “embodied intelligence” has sparked a new wave of research and technology, leading AI beyond the digital realm into a new paradigm that can actively act and perceive in a physical environment through tangible entities such as robots and automated devices [5].

The latest developments in parallel driving foreshadow the possibility of delivering intelligence across organizations using foundation models. As is well-known, there are limitations in scenario acquisition in the field of intelligent vehicles (IV), such as efficiency, diversity, and complexity, which hinder in-depth research of vehicle intelligence. To address this issue, this manuscript draws inspiration from scenarios engineering, parallel driving and introduces a pioneering framework for scenario generation, leveraging the ChatGPT, denoted as SeGPT. Within this framework, we define a trajectory scenario and design prompts engineering to generate complex and challenging scenarios. Furthermore, SeGPT, in combination with “Three Modes”, foundation models, vehicle operating system, and other advanced infrastructure, holds the potential to achieve higher levels of autonomous driving. Experimental outcomes substantiate SeGPT's adeptness in producing a spectrum of varied scenarios, underscoring its potential to augment the development of trajectory prediction algorithms. These findings mark significant progress in the domain of scenario generation, also pointing towards new directions in the research of vehicle intelligence and scenarios engineering.

Long-tail effect, characterized by highly frequent occurrence of normal scenarios and the scarce appearance of extreme “long-tail” scenarios, ubiquitously exists in the vision-related problems in the real-world applications. Though many computer vision methods to date have already achieved feasible performance for most of the normal scenarios, it is still challenging for existing vision systems to accurately perceive the long-tail scenarios. This deficiency largely hinders the practical application of computer vision systems, since long-tail problems may incur fatal consequences, such as traffic accidents, taking the vision systems of autonomous vehicles as an example. In this paper, we firstly propose a theoretical framework named Long-tail Regularization (LoTR), for analyzing and tackling the long-tail problems in the vision perception of autonomous driving. LoTR is able to regularize the scarcely occurred long-tail scenarios to be frequently encountered. Then we present a Parallel Vision Actualization System (PVAS), which consists of closed-loop optimization and virtual-real interaction, to search for challenging long-tail scenarios and produce large-scale long-tail driving scenarios for autonomous vehicles. In addition, we introduce how to perform PVAS in Intelligent Vehicle Future Challenge of China (IVFC), the most durable autonomous driving competition around the world. Results over the past decade demonstrate that PVAS can effectively guide the collection of long-tail data to diminish the cost in the real world, and thus promote the capability of vision systems to adapt to complex environments, alleviating the impact of long-tail effect.

Estimating absolute 3D poses of multiple people from monocular image is challenging due to the presence of occlusions and the scale variation among different persons. Among the existing methods, the top-down paradigms are highly dependent on human detection which is prone to the influence from inter-person occlusions, while the bottom-up paradigms suffer from the difficulties in keypoint feature extraction caused by scale variation and unreliable joint grouping caused by occlusions. To address these challenges, we introduce a novel multi-person 3D pose estimation framework, aided by multi-scale feature representations and human depth perceiving. Firstly, a waterfall-based architecture is incorporated for multi-scale feature representations to achieve a more accurate estimation of occluded joints with a better detection of human shapes. Then the global and local representations are fused for handling the effects of inter-person occlusion and scale variation in depth perceiving and keypoint feature extraction. Finally, with the guidance of the fused multi-scale representations, a depth-aware model is exploited for better 2D joint grouping and 3D pose recovering. Quantitative and qualitative evaluations on benchmark datasets of MuCo-3DHP and MuPoTS-3D prove the effectiveness of our proposed method. Furthermore, we produce an occluded MuPoTS-3D dataset and the experiments on it validate the superiority of our method for overcoming the occlusions.

Lecturers, as the guidance of the classroom, play a significant role in the teaching process. However, the lecturers’ sense of space immersion has been ignored in current virtual teaching systems. In this article, we explore the cyber–physical–social intelligence for Edu-Metaverse in cyber–physical–social space and specially design a lecturer-centered immersive teaching system, taking the social and lecturers’ factors into consideration. We call this system VirtualClassroom (V-Classroom). Specifically, we first introduce the cyber–physical–social system (CPSS) paradigm of V-Classroom so that the workflow is standardized and significantly simplified, and the systems can be constructed with off-the-shelf hardware. The key component of V-Classroom is a cyber-world representation of a physical-world classroom instrumented with sparse consumer-grade RGBD cameras for capturing the 3-D geometry and texture of the classrooms. We provide each V-Classroom lecturer with a physical device for sending 6DoF view-change messages and showing view-dependent content of the remote classroom. Following the above paradigm, we develop the V-Classroom algorithms, including V-Classroom depth algorithm (V-DA) and V-Classroom view algorithm (V-VA), to achieve the real-time rendering of remote classrooms. V-DA is dedicated to recovering accurate depth information of the classrooms while V-VA is devoted to real-time novel view synthesis. Finally, we illustrate our implemented CPSS-driven V-Classroom prototype, based on real-world classroom scenarios we collected, and discuss the main challenges and future direction.