Qionghai Dai

• Tsinghua University

Invasive infections by encapsulated bacteria are the major cause of human morbidity and mortality. The liver resident macrophages, Kupffer cells, form the hepatic firewall to clear many encapsulated bacteria in the blood circulation but fail to control certain high-virulence capsule types. Here we report that the spleen is the backup immune organ t...

Super-resolution (SR) neural networks transform low-resolution optical microscopy images into SR images. Application of single-image SR (SISR) methods to long-term imaging has not exploited the temporal dependencies between neighboring frames and has been subject to inference uncertainty that is difficult to quantify. Here, by building a large-scal...

Quantitative ethology necessitates accurate tracking of animal locomotion, especially for population-level analyses involving multiple individuals. However, current methods rely on laborious annotations for supervised training and have restricted performance in challenging conditions. Here we present an unsupervised deep-learning method for multi-a...

The concomitant development and evolution of lung computed tomography (CT) and artificial intelligence (AI) has allowed non-invasive lung imaging to be a key part of the clinical care of patients with major diseases, such as lung cancer. However, the paucity of labeled lung CT data has limited the training highly efficacious AI models and thereby h...

This roadmap consolidates recent advances while exploring emerging applications, reflecting the remarkable diversity of hardware platforms, neuromorphic concepts, and implementation philosophies reported in the field. It emphasizes the critical role of cross-disciplinary collaboration in this rapidly evolving field.

The integration of non-invasive brain imaging techniques, particularly computed tomography (CT) and magnetic resonance imaging (MRI), coupled with the advancement of artificial intelligence, is forging a key pathway for brain disease diagnosis, playing a vital role in safeguarding human health. A robust artificial intelligence copilot is essential...

Recent advances in deep learning (DL) have prompted the development of high-performing early warning score (EWS) systems, predicting clinical deteriorations such as acute kidney injury, acute myocardial infarction, or circulatory failure. DL models have proven to be powerful tools for various tasks but come with the cost of lacking interpretability...

Camera-based single-molecule techniques have emerged as crucial tools in revolutionizing the understanding of biochemical and cellular processes due to their ability to capture dynamic processes with high precision, high-throughput capabilities, and methodological maturity. However, the stringent requirement in photon number per frame and the limit...

Multi-modal image fusion aims to amalgamate pivotal information from various sensor sources to provide informative visual representation in imaging scenes. Rapid and precise fusion of images is crucial for practical applications in fields such as autonomous driving and medical diagnostics. However, the primary challenge lies in balancing computatio...

Gaussian splatting has gained attention for its efficient representation and rendering of 3D scenes using continuous Gaussian primitives. However, it struggles with sparse-view inputs due to limited geometric and photometric information, causing ambiguities in depth, shape, and texture. we propose GBR: Generative Bundle Refinement, a method for hig...

Large-scale neural recording with single-neuron resolution has revealed the functional complexity of the neural systems. However, even under well-designed task conditions, the cortex-wide network exhibits highly dynamic trial variability, posing challenges to the conventional trial-averaged analysis. To study mesoscale trial variability, we conduct...

Video snapshot compressive imaging (SCI) encodes the target dynamic scene compactly into a snapshot and reconstructs its high-speed frame sequence afterward, greatly reducing the required data footprint and transmission bandwidth as well as enabling high-speed imaging with a low frame rate intensity camera. In implementation, high-speed dynamics ar...

The neural radiance field (NeRF) achieved remarkable success in modeling 3D scenes and synthesizing high-fidelity novel views. However, existing NeRF-based methods focus more on making full use of high-resolution images to generate high-resolution novel views, but less considering the generation of high-resolution details given only low-resolution...

Deep learning (DL) has ushered in a suite of promising tools for image processing, including denoising (DN), deblurring (DB), and super-resolution (SR). However, traditional DL methods assume independent and identically distributed (i.i.d.) data for model training and inference, which does not hold in practice due to factors such as sample variatio...

In toto imaging of large-scale transparent samples or cleared tissue is in high demand in broad biological applications such as oncology, neuroscience, and developmental biology to understand the functions and organizations of large-scale cells at organ level. However, traditional methods usually face resolution degradation due to the missing cone...

This chapter presents a novel microscopy technique, real-time ultra-large scale imaging with high-resolution microscopy (RUSH), which enables imaging of biological systems with unprecedented space–bandwidth product and data throughput. We describe the design and implementation of the RUSH system, which comprises a customized objective lens, a camer...

Computational imaging, as a novel technology utilizing encoded image acquisition, relies on intelligent decoding methods for effective image restoration and sensing. Optical computing‐based decoders can efficiently process and extract features from pre‐sensor information, reducing the computational burden on digital computers. However, mainstream p...

Depth sensing plays a crucial role in various applications, including robotics, augmented reality, and autonomous driving. Monocular passive depth sensing techniques have come into their own for the cost-effectiveness and compact design, offering an alternative to the expensive and bulky active depth sensors and stereo vision systems. While the lig...

The demand for compact cameras capable of recording high-speed scenes with high resolution is steadily increasing. However, achieving such capabilities often entails high bandwidth requirements, resulting in bulky, heavy systems unsuitable for low-capacity platforms. To address this challenge, leveraging a coded exposure setup to encode a frame seq...

This paper proposes a hybrid radiance field representation for unbounded immersive light field reconstruction which supports high-quality rendering and aggressive view extrapolation. The key idea is to first formally separate the foreground and the background and then adaptively balance learning of them during the training process. To fulfill this...

Optical computing promises to improve the speed and energy efficiency of machine learning applications1–6. However, current approaches to efficiently train these models are limited by in silico emulation on digital computers. Here we develop a method called fully forward mode (FFM) learning, which implements the compute-intensive training process o...

Recent advances in spatial omics have expanded the spectrum of profiled molecular categories beyond transcriptomics. However, many of these technologies are constrained by limited spatial resolution, hindering our ability to deeply characterize intricate tissue architectures. Existing computational methods primarily focus on the resolution enhancem...

Calculus equations serve as fundamental frameworks in mathematics, enabling describing an extensive range of natural phenomena and scientific principles, such as thermodynamics and electromagnetics. Analog computing with electromagnetic waves presents an intriguing opportunity to solve calculus equations with unparalleled speed, while facing an ine...

Three-dimensional (3D) perception is vital to drive mobile robotics’ progress toward intelligence. However, state-of-the-art 3D perception solutions require complicated postprocessing or point-by-point scanning, suffering computational burden, latency of tens of milliseconds, and additional power consumption. Here, we propose a parallel all-optical...

Efficient storage and sharing of massive biomedical data would open up their wide accessibility to different institutions and disciplines. However, compressors tailored for natural photos/videos are rapidly limited for biomedical data, while emerging deep learning–based methods demand huge training data and are difficult to generalize. Here, we pro...

Turbulence is a complex and chaotic state of fluid motion. Atmospheric turbulence within the Earth’s atmosphere poses fundamental challenges for applications such as remote sensing, free-space optical communications and astronomical observation due to its rapid evolution across temporal and spatial scales. Conventional methods for studying atmosphe...

Single image super-resolution (SISR) neural networks for optical microscopy have shown great capability to directly transform a low-resolution (LR) image into its super-resolution (SR) counterpart, enabling low-cost long-term live-cell SR imaging. However, when processing time-lapse data, current SISR models failed to exploit the important temporal...

Exploring the relationship between neuronal dynamics and ethologically relevant behaviour involves recording neuronal-population activity using technologies that are compatible with unrestricted animal behaviour. However, head-mounted microscopes that accommodate weight limits to allow for free animal behaviour typically compromise field of view, r...

Lattice light-sheet microscopy (LLSM) provides a crucial observation window into intra- and inter-cellular physiology of living specimens with high speed and low phototoxicity, however, at the diffraction-limited resolution or anisotropic super-resolution with structured illumination. Here we present the meta-learning-empowered reflective lattice l...

Long-term observation of subcellular dynamics in living organisms is limited by background fluorescence originating from tissue scattering or dense labeling. Existing confocal approaches face an inevitable tradeoff among parallelization, resolution and phototoxicity. Here we present confocal scanning light-field microscopy (csLFM), which integrates...

In the field of medical image compression, Implicit Neural Representation (INR) networks have shown remarkable versatility due to their flexible compression ratios, yet they are constrained by a one-to-one fitting approach that results in lengthy encoding times. Our novel method, ``\textbf{UniCompress}'', innovatively extends the compression capabi...

Computational super-resolution methods, including conventional analytical algorithms and deep learning models, have substantially improved optical microscopy. Among them, supervised deep neural networks have demonstrated outstanding performance, however, demanding abundant high-quality training data, which are laborious and even impractical to acqu...

Observing the dynamics of colorless or transparent samples quantitatively is vital in cellular-level biomedical research. Among existing techniques, fluorescent microscopy suffers from phototoxicity and photobleaching issues, while snapshot quantitative phase imaging (QPI) demands a complicated setup. Thus, an easily accessible tool allowing for lo...

Lattice light-sheet microscopy (LLSM) provides a crucial observation window into intra- and inter-cellular physiology of living specimens with high speed and low phototoxicity, however, at the diffraction-limited resolution or anisotropic super-resolution with structured illumination. Here we present the meta learning-empowered reflective lattice l...

Single image super-resolution (SISR) neural networks for optical microscopy have shown great capability to directly transform a low-resolution (LR) into its super-resolution (SR) counterpart, enabling low-cost long-term live-cell SR imaging. However, when processing time-lapse data, current SISR models failed to exploit the important temporal depen...

The spleen is well-known for defense against invasive infections of encapsulated bacteria, particularly Streptococcus pneumoniae (pneumococcus). However, the precise mechanism of the splenic anti-bacterial immunity remains elusive. Here we report that red pulp (RP) macrophages execute the splenic defense against S. pneumoniae in mice, with the help...

The pursuit of artificial general intelligence (AGI) continuously demands higher computing performance. Despite the superior processing speed and efficiency of integrated photonic circuits, their capacity and scalability are restricted by unavoidable errors, such that only simple tasks and shallow models are realized. To support modern AGIs, we des...

Causal discovery in time-series is a fundamental problem in the machine learning community, enabling causal reasoning and decision-making in complex scenarios. Recently, researchers successfully discover causality by combining neural networks with Granger causality, but their performances degrade largely when encountering high-dimensional data beca...

Event cameras or dynamic vision sensors (DVS) record asynchronous response to brightness changes instead of conventional intensity frames, and feature ultra-high sensitivity at low bandwidth. The new mechanism demonstrates great advantages in challenging scenarios with fast motion and large dynamic range. However, the recorded events might be highl...

Medical artificial intelligence (AI) offers great potential for automatic pathology interpretation, but the performance is far behind providing a practical tool in clinical settings, which demands both pixel-level accuracy and high interpretability for diagnosis. The main challenges lie in that the construction of such AI models relies on substanti...

Understanding how neuronal dynamics gives rise to ethologically relevant behavior requires recording of neuronal population activity via technologies that are compatible with unconstrained animal behavior. However, realizations of cellular resolution head-mounted microscopes for mice have been based on conventional microscope designs that feature v...

Multimode fiber (MMF) which supports parallel transmission of spatially distributed information is a promising platform for remote imaging and capacity-enhanced optical communication. However, the variability of the scattering MMF channel poses a challenge for achieving long-term accurate transmission over long distances, of which static optical pr...

Visual sensors are indispensable for automatic vehicles, to achieve comprehensive environmental perception for navigation, but their deteriorated performance in harsh illuminations largely sets back the practical use of autonomous driving technologies. A promising solution is to use a bio-inspired event sensor that asynchronously records the intens...

We present Raw2raw, an innovative image reconstruction pipeline to mitigate blurriness caused by under-optimized smartphone systems. Our style effectively preserves data structure and underscores the potential of portable biomedical data capture and smart-phone camera applications.

Harnessing the two-photon synthetic aperture microscopy (2pSAM), we achieved high-quality 3D reconstruction, even with quadruple downsampling. This method fully utilizes the information redundancy inherent in four-dimensional spatial and angular scanning.

We combine line confocal scheme and scanning light-field microscopy (sLFM) to develop a confocal scanning light-field microscopy (csLFM). By introducing simple modifications in the illumination path, csLFM enables high-resolution imaging with reduction in background.

Fluorescence imaging with high signal-to-noise ratios has become the foundation of accurate visualization and analysis of biological phenomena. However, the inevitable noise poses a formidable challenge to imaging sensitivity. Here we provide the spatial redundancy denoising transformer (SRDTrans) to remove noise from fluorescence images in a self-...

Fluorescence microscopy allows for the high-throughput imaging of cellular activity across brain areas in mammals. However, capturing rapid cellular dynamics across the curved cortical surface is challenging, owing to trade-offs in image resolution, speed, field of view and depth of field. Here we report a technique for wide-field fluorescence imag...

Cardiac surgery-associated Acute Kidney Injury (CSA-AKI) is a significant complication that often leads to increased morbidity and mortality. Effective CSA-AKI management relies on timely diagnosis and interventions. However, many cases of CSA-AKI are detected too late. Despite the efforts of novel biomarkers and data-driven predictive models, thei...

Cardiac surgery-associated Acute Kidney Injury (CSA-AKI) is a significant complication that often leads to increased morbidity and mortality. Effective CSA-AKI management relies on timely diagnosis and interventions. However, many cases of CSA-AKI are detected too late. Despite the efforts of novel biomarkers and data-driven predictive models, thei...

Signal capture is at the forefront of perceiving and understanding the environment; thus, imaging plays a pivotal role in mobile vision. Recent unprecedented progress in artificial intelligence (AI) has shown great potential in the development of advanced mobile platforms with new imaging devices. Traditional imaging systems based on the “capturing...

The traditional 3D object retrieval (3DOR) task is under the close-set setting, which assumes the categories of objects in the retrieval stage are all seen in the training stage. Existing methods under this setting may tend to only lazily discriminate their categories, while not learning a generalized 3D object embedding. Under such circumstances,...

Fluorescence microscopy has become an indispensable tool for revealing the dynamic regulation of cells and organelles. However, stochastic noise inherently restricts optical interrogation quality and exacerbates observation fidelity when balancing the joint demands of high frame rate, long-term recording and low phototoxicity. Here we propose DeepS...

Functional imaging of biological dynamics generally begins with acquiring time-series images, followed by quantifying spatially averaged intensity traces for the regions of interest (ROIs). The conventional pipeline discards a substantial portion of the acquired data when quantifying intensity traces, indicative of inefficient data acquisition. Her...

Big data serves as the cornerstone for constructing real-world deep learning systems across various domains. In medicine and healthcare, a single clinical site lacks sufficient data, thus necessitating the involvement of multiple sites. Unfortunately, concerns regarding data security and privacy hinder the sharing and reuse of data across sites. Ex...

Photonic computing enables faster and more energy-efficient processing of vision data 1–5 . However, experimental superiority of deployable systems remains a challenge because of complicated optical nonlinearities, considerable power consumption of analog-to-digital converters (ADCs) for downstream digital processing and vulnerability to noises and...

This paper proposes a hybrid radiance field representation for unbounded immersive light field reconstruction which supports high-quality rendering and aggressive view extrapolation. The key idea is to first formally separate the foreground and the background and then adaptively balance learning of them during the training process. To fulfill this...

Background: Mediastinal neoplasms are typical thoracic diseases with increasing incidence in the general global population and can lead to poor prognosis. In clinical practice, the mediastinum's complex anatomic structures and intertype confusion among different mediastinal neoplasm pathologies severely hinder accurate diagnosis. To solve these di...

Multi-spectral imaging is a fundamental tool characterizing the constituent energy of scene radiation. However, current multi-spectral video cameras cannot scale up beyond megapixel resolution due to optical constraints and the complexity of the reconstruction algorithms. To circumvent the above issues, we propose a tens-of-megapixel handheld multi...

Computational super-resolution (SR) methods, including conventional analytical algorithms and deep learning models, have substantially improved optical microscopy. Among them, supervised deep neural networks have demonstrated outstanding SR performance, however, demanding abundant high-quality training data, which are laborious and even impractical...

Recent years have witnessed remarkable achievements in video-based action recognition. Apart from traditional frame-based cameras, event cameras are bio-inspired vision sensors that only record pixel-wise brightness changes rather than the brightness value. However, little effort has been made in event-based action recognition, and large-scale publ...

Deep learning has yielded promising results for medical image diagnosis but relies heavily on manual image annotations, which are expensive to acquire. We present Cross-DL, a cross-modality learning framework for intracranial abnormality detection and localization in head computed tomography (CT) scans by learning from free-text imaging reports. Cr...

Coded exposure photography is a promising computational imaging technique capable of addressing motion blur much better than using a conventional camera, via tailoring invertible blur kernels. However, existing methods suffer from restrictive assumptions, complicated preprocessing, and inferior performance. To address these issues, we proposed an e...

Genomic and transcriptomic image data, represented by DNA and RNA fluorescence in situ hybridization (FISH), respectively, together with proteomic data, particularly that related to nuclear proteins, can help elucidate gene regulation in relation to the spatial positions of chromatins, messenger RNAs, and key proteins. However, methods for image-ba...

Structured illumination microscopy (SIM) has become the standard for next-generation wide-field microscopy, offering ultrahigh imaging speed, superresolution, a large field-of-view, and long-term imaging. Over the past decade, SIM hardware and software have flourished, leading to successful applications in various biological questions. However, unl...

The optical microscope is customarily an instrument of substantial size and expense but limited performance. Here we report an integrated microscope that achieves optical performance beyond a commercial microscope with a 5×, NA 0.1 objective but only at 0.15 cm³ and 0.5 g, whose size is five orders of magnitude smaller than that of a conventional m...

Along with broad applications of the linear Doppler effect, the rotational Doppler effect (RDE) of a structured light source carrying orbital angular momentum (OAM) has attracted significant attention for applications ranging from optical sensors to Doppler cooling. However, the high‐purity structured source's low energy efficiency and unknown opti...

Massive collection and explosive growth of biomedical data, demands effective compression for efficient storage, transmission and sharing. Readily available visual data compression techniques have been studied extensively but tailored for natural images/videos, and thus show limited performance on biomedical data which are of different features and...

Fluorescence imaging with high signal-to-noise ratios has become the foundation of accurate visualization and analysis of biological phenomena. However, the inevitable photon shot noise poses a formidable challenge on imaging sensitivity. In this paper, we provide a spatial redundancy denoising transformer (SRDTrans) to remove noise from fluorescen...

Video object detection is a widely studied topic and has made significant progress in the past decades. However, the feature extraction and calculations in existing video object detectors demand decent imaging quality and avoidance of severe motion blur. Under extremely dark scenarios, due to limited sensor sensitivity, we have to trade off signal-...

Solving partial differential equations (PDEs) has been a fundamental problem in computational science and of wide applications for both scientific and engineering research. Due to its universal approximation property, neural network is widely used to approximate the solutions of PDEs. However, existing works are incapable of solving high-order PDEs...

Causal discovery in time-series is a fundamental problem in the machine learning community, enabling causal reasoning and decision-making in complex scenarios. Recently, researchers successfully discover causality by combining neural networks with Granger causality, but their performances degrade largely when encountering high-dimensional data beca...

The neural radiance field (NeRF) achieved remarkable success in modeling 3D scenes and synthesizing high-fidelity novel views. However, existing NeRF-based methods focus more on the make full use of the image resolution to generate novel views, but less considering the generation of details under the limited input resolution. In analogy to the exte...

Multimode fiber (MMF) which supports parallel transmission of spatially distributed information is a promising platform for remote imaging and capacity-enhanced optical communication. However, the variability of the scattering MMF channel poses a challenge for achieving long-term accurate transmission over long distances, of which static optical pr...

Imagine an interesting multimodal interactive scenario that you can see, hear, and chat with an AI-generated digital character, who is capable of behaving like Sheldon from The Big Bang Theory, as a DEEP copy from appearance to personality. Towards this fantastic multimodal chatting scenario, we propose a novel task, named Deep Personalized Charact...