Figure - available from: Nature Methods
This content is subject to copyright. Terms and conditions apply.
Training data augmentation through random smooth elastic deformation (a) Upper left: Raw image; Upper right: Labels; Lower Left: Loss Weights; Lower Right: 20μm grid (for illustration purpose only) (b) Deformation field (black arrows) generated using bicubic interpolation from a coarse grid of displacement vectors (blue arrows; magnification: 5×). Vector components are drawn from a Gaussian distribution (σ = 10px). (c) Backwarp-transformed images of (a) using the deformation field
Source publication
U-Net is a generic deep-learning solution for frequently occurring quantification tasks such as cell detection and shape measurements in biomedical image data. We present an ImageJ plugin that enables non-machine-learning experts to analyze their data with U-Net on either a local computer or a remote server/cloud service. The plugin comes with pret...
Citations
... The U-Net plugin [155] stands out as a DL software dedicated to cell counting, detection, and morphometry. Additionally, 3DeeCellTracker [156] introduces a DL-based pipeline designed for segmenting and tracking cells in 3D time-lapse images. ...
Microscopic image segmentation (MIS) is a fundamental task in medical imaging and biological research, essential for precise analysis of cellular structures and tissues. Despite its importance, the segmentation process encounters significant challenges, including variability in imaging conditions, complex biological structures, and artefacts (e.g., noise), which can compromise the accuracy of traditional methods. The emergence of deep learning (DL) has catalyzed substantial advancements in addressing these issues. This systematic literature review (SLR) provides a comprehensive overview of state-of-the-art DL methods developed over the past six years for the segmentation of microscopic images. We critically analyze key contributions, emphasizing how these methods specifically tackle challenges in cell, nucleus, and tissue segmentation. Additionally, we evaluate the datasets and performance metrics employed in these studies. By synthesizing current advancements and identifying gaps in existing approaches, this review not only highlights the transformative potential of DL in enhancing diagnostic accuracy and research efficiency but also suggests directions for future research. The findings of this study have significant implications for improving methodologies in medical and biological applications, ultimately fostering better patient outcomes and advancing scientific understanding.
... We used U-net schemes as described in Ronneberger et al. (2015). These schemes are among the state-of-the-art neural 120 architectures for mapping problems with n-dimensional tensors, with numerous applications in imaging science (Falk et al., 2019), as well as recent applications in ocean science (Lguensat et al., 2018;Beauchamp et al., 2022;Jenkins et al., 2022). For each training run, we use 8604 Lagrangian experiments for training, 1224 for validation, and 6800 for testing. ...
The gravitational pump plays a key role in the ocean carbon cycle by exporting sinking organic carbon from the surface to the deep ocean. Deep sediment trap time-series provide unique measurements of this sequestered carbon flux. Sinking particles are influenced by physical short-term spatio-temporal variability, which inhibits the establishment of a direct link to their surface origin. In this study, we present a novel machine learning tool, designated as Unetsst-ssh, which is capable of predicting the catchment area of particles captured by sediment traps moored at a depth of 3000 m above the Porcupine Abyssal Plain (PAP), based solely on surface data. The machine learning tool was trained and evaluated using Lagrangian experiments in a realistic CROCO numerical simulation. The conventional approach of assuming a static 100–200 km box over the sediment trap location, only yields an average prediction of ∼25 % of the source region, whilst Unetsst-ssh predicts ∼50 %. Unetsst-ssh was then applied to satellite observations to create a 20-year catchment area dataset, which demonstrates a stronger correlation between the PAP site deep particle fluxes and surface chlorophyll concentration, compared with the conventional approach. However, predictions remain highly sensitive to the local deep dynamics which are not observed in surface ocean dynamics. The improved identification of the particle source region for deep ocean sediment traps can facilitate a more comprehensive understanding of the mechanisms driving the export of particles from the surface to the deep ocean, a key component of the biological carbon pump.
... The architecture of UNet is shown in Fig. 2 and is mostly used for biomedical image semantic segmentation [51]. The visual shape of UNet looks like the alphabet 'U' , so it is called UNet. ...
... The used number of channels is indicated on the top of the line, and the lower left edge of the line indicates x-y-size. The arrows indicate different operations[51] ...
Detection of cancer in human organs at an early stage is a crucial task and is important for the survival of the patients, especially in terms of complex structure, dynamic size, and dynamic length in organs like the pancreas. To deal with this problem, pancreatic semantic segmentation was introduced, but it was hampered by challenges related to image modalities and the availability of limited datasets. This paper provides different deep learning models for pancreatic detection. The proposed model pipeline has two phases: pancreas localization and segmentation. In the first phase, rough regions of the pancreas are detected with YOLOv5, and the detected regions are cropped to avoid an imbalance between the pancreas region and the background. In the second phase, the detected regions are segmented with various models like UNet, VNet, SegResNet and HighResNet for effective detection of cancer regions. The experiments were conducted on a private dataset collected from the Champalimaud Foundation in Portugal. The model’s performance is evaluated in terms of quantitative and qualitative analysis. From experiments, we found that, when compared to other Nets, YOLOv5 is superior in pancreatic area localization and 2.5D HighResNet is superior in segmentation.
... A method for data association in particle tracking is explained in [110]. In another study, [111] utilized a vanilla U-Net model for cell counting, detection, and morphometry. This work, also presented as a plugin for ImageJ, allows researchers without deep learning expertise to benefit from U-Net in biological image discovery. ...
Object tracking serves as a cornerstone of modern technological innovation, with applications spanning diverse fields such as defense systems, autonomous vehicles, and the cutting edge of biomedical research. Fundamentally, it involves the precise identification, monitoring, and spatiotemporal analysis of objects across sequential frames, enabling a deeper understanding of dynamic behaviors. In cell biology, this capability is indispensable for unraveling the intricacies of cellular mechanisms—offering insights into cell migration, interactions, and responses to external stimuli like drugs or pathogens. These insights not only illuminate fundamental biological processes but also pave the way for breakthroughs in understanding disease progression and therapeutic interventions.
Over the years, object tracking methodologies have evolved significantly, progressing from traditional feature-based approaches—leveraging color, shape, and edges—to advanced machine learning and deep learning frameworks. While classical methods demonstrate reliability under controlled conditions, they falter in complex environments characterized by occlusions, variable lighting, and high object density. In contrast, modern deep learning models excel in such challenging scenarios, offering unparalleled accuracy, adaptability, and robustness.
This paper presents a comprehensive review of object tracking techniques, systematically categorizing them into traditional, statistical, feature-based, and machine learning paradigms. We place a particular emphasis on their applications in biomedical research, where precise tracking of cells and subcellular structures is critical for advancing our understanding of health and disease. Key performance metrics, including accuracy, computational efficiency, and adaptability, are examined to facilitate meaningful comparisons across methodologies.
Despite remarkable advancements, the field still faces a pivotal challenge: Why does the development of a fully integrated, robust, and scalable end-to-end tracking system capable of handling diverse biomedical scenarios remain elusive? Addressing this question, we identify the limitations of current technologies and explore emerging trends poised to overcome these barriers. Our goal is to provide a roadmap for the development of next-generation object tracking systems—tools that will not only transform biomedical research but also catalyze innovations across broader scientific and technological domains.
... Detection-based methods refer to the enumeration of instances once they are located in the image, being image segmentation as the most accurate method of instance detection. [24], [11], [25] extend the U-Net architecture [26] in its ability to count cells and bacteria by discretizing them from background and subsequently enumerating them. Regression-based methods solve the task either through density map estimation or direct regression without spatial details. ...
Microorganism enumeration is an essential task in many applications, such as assessing contamination levels or ensuring health standards when evaluating surface cleanliness. However, it's traditionally performed by human-supervised methods that often require manual counting, making it tedious and time-consuming. Previous research suggests automating this task using computer vision and machine learning methods, primarily through instance segmentation or density estimation techniques. This study conducts a comparative analysis of vision transformers (ViTs) for weakly-supervised counting in microorganism enumeration, contrasting them with traditional architectures such as ResNet and investigating ViT-based models such as TransCrowd. We trained different versions of ViTs as the architectural backbone for feature extraction using four microbiology datasets to determine potential new approaches for total microorganism enumeration in images. Results indicate that while ResNets perform better overall, ViTs performance demonstrates competent results across all datasets, opening up promising lines of research in microorganism enumeration. This comparative study contributes to the field of microbial image analysis by presenting innovative approaches to the recurring challenge of microorganism enumeration and by highlighting the capabilities of ViTs in the task of regression counting.
... Therefore, we used a deep learning method incorporating a U-net-based convolutional neural network (CNN) 15 for automatic brain tumor segmentation directly from conventional MRI-acquired images, including the core and peritumor edema regions of the tumor and constructed a prediction model for the subtypes of glioma with machine learning techniques based on radiological features from the tumor core and edema, and explore its molecular pathological characteristics. ...
Comprehensive and non-invasive preoperative molecular diagnosis is important for prognostic and therapy decision-making in adult-type diffuse gliomas. We employed a deep learning method for automatic segmentation of brain gliomas directly from conventional magnetic resonance imaging (MRI) scans of the tumor core and peritumoral edema regions based on available glioma MRI data provided in the BraTS2021. Three-dimensional volumes of interest were segmented from 424 cases of glioma imaging data retrospectively obtained from two medical centers using the segmentation method and radiomic features were extracted. We developed a subtype prediction model based on extracted radiomic features and analyzed significance and correlations between glioma morphological characteristics and pathological features using data from patients with adult-type diffuse glioma. The automated segmentation achieved mean Dice scores of 0.884 and 0.889 for the tumor core and whole tumor, respectively. The area under the receiver operating characteristic curve for the prediction of adult-type diffuse gliomas subtypes was 0.945. “Glioblastoma, IDH-wildtype”, “Astrocytoma, IDH-mutant”, and “Oligodendroglioma, IDH-mutant, 1p/19q-coded” showed AUCs of 0.96, 0.914, and 0.961, respectively, for subtype prediction. Glioma morphological characteristics, molecular and pathological levels, and clinical data showed significant differences and correlations. An automatic segmentation model for gliomas based on 3D U-Nets was developed, and the prediction model for gliomas built using the parameters obtained from the automatic segmentation model showed high overall performance.
Supplementary Information
The online version contains supplementary material available at 10.1038/s41598-024-79344-9.
... Moreover, the combination of stains provided clear differentiation of organs (i.e. pronephros, neural tissue, developing eyes) (Fig. 3D, i and vii) in developing Xenopus embryos, serving as a robust dataset for training supervised deep learning networks, such as U-Net 47,48 . We extracted local three-dimensional context from each eye (Fig. 3D, ii and viii) and trained Mab-Net, our deep learning solution, for automated segmentation of the retina (Fig. 3D, vi and xii, cyan) and lens (Fig. 3D, vi and xii, yellow), facilitating straightforward feature extraction and automated size measurements. ...
... From mesoSPIM recordings, subvolumes (300x300xdepth) containing eyes were manually extracted. Network training was on a classical 2D U-Net architecture using the model architecture of 2D-CellNet and the U-Net Fiji plug-in 47,48 . Intersection Over Union (IOU) reported are as calculated by the U-Net Fiji plug-in, and a tile size of 508 × 508 pixels was used. ...
Anophthalmia, microphthalmia and coloboma (AMC) comprise a spectrum of developmental eye disorders, accounting for approximately 20% of childhood visual impairment. While non-coding regulatory sequences are increasingly recognised as contributing to disease burden, characterising their impact on gene function and phenotype remains challenging. Furthermore, little is known of the nature and extent of their contribution to AMC phenotypes. We report two families with variants in or near MAB21L2, a gene where genetic variants are known to cause AMC in humans and animal models. The first proband, presenting with microphthalmia and coloboma, has a likely pathogenic missense variant (c.338 G > C; p.[Trp113Ser]), segregating within the family. The second individual, presenting with microphthalmia, carries an ~ 113.5 kb homozygous deletion 19.38 kb upstream of MAB21L2. Modelling of the deletion results in transient small lens and coloboma as well as midbrain anomalies in zebrafish, and microphthalmia and coloboma in Xenopus tropicalis. Using conservation analysis, we identify 15 non-coding conserved elements (CEs) within the deleted region, while ChIP-seq data from mouse embryonic stem cells demonstrates that two of these (CE13 and 14) bind Otx2, a protein with an established role in eye development. Targeted disruption of CE14 in Xenopus tropicalis recapitulates an ocular coloboma phenotype, supporting its role in eye development. Together, our data provides insights into regulatory mechanisms underlying eye development and highlights the importance of non-coding sequences as a source of genetic diagnoses in AMC.
... In other words, objects that exhibit visible boundaries from their centers are effectively predicted using StarDist [17]. The algorithm relies on U-Net based framework specifically designed to segment nuclei and cells in microscopy images, as the model is pre-trained on diverse fluorescent microscopy images of nuclei [18]. The model integrates object probability prediction and distances to the object boundary along a predetermined set of radial directions referred to as rays. ...
With the recent surge in the development of highly selective probes, fluorescence microscopy has become one of the most widely used approaches to studying cellular properties and signaling in living cells and tissues. Traditionally, microscopy image analysis heavily relies on manufacturer-supplied software, which often demands extensive training and lacks automation capabilities for handling diverse datasets. A critical challenge arises if the fluorophores employed exhibit low brightness and a low signal-to-noise ratio (SNR). Consequently, manual intervention may become a necessity, introducing variability in the analysis outcomes even for identical samples when analyzed by different users. This leads to the incorporation of blinded analysis, which ensures that the outcome is free from user bias to a certain extent but is extremely time-consuming. To overcome these issues, we developed a tool called DL-SCAN that automatically segments and analyzes fluorophore-stained regions of interest such as cell bodies in fluorescence microscopy images using deep learning. We demonstrate the program’s ability to automate cell identification and study cellular ion dynamics using synthetic image stacks with varying SNR. This is followed by its application to experimental Na+ and Ca2+ imaging data from neurons and astrocytes in mouse brain tissue slices exposed to transient chemical ischemia. The results from DL-SCAN are consistent, reproducible, and free from user bias, allowing efficient and rapid analysis of experimental data in an objective manner. The open-source nature of the tool also provides room for modification and extension to analyze other forms of microscopy images specific to the dynamics of different ions in other cell types.
... Attributed to their ability to engage in hierarchical learning, these techniques are proficient in depicting complex nonlinear associations [20], which are instrumental in tasks such as categorization, the integration of data, and the reduction in dimensions [21][22][23]. In the domain of land cover classification, deep learning has achieved promising outcomes, particularly with models like U-Net [24], capable of yielding robust classification outcomes despite there being a smaller dataset for training. Several investigations have utilized adjusted loss functions and augmentation strategies to enhance model resilience against class imbalance issues in datasets [25]. ...
The prompt acquisition of precise land cover categorization data is indispensable for the strategic development of contemporary farming practices, especially within the realm of forestry oversight and preservation. Forests are complex ecosystems that require precise monitoring to assess their health, biodiversity, and response to environmental changes. The existing methods for classifying remotely sensed imagery often encounter challenges due to the intricate spacing of feature classes, intraclass diversity, and interclass similarity, which can lead to weak perceptual ability, insufficient feature expression, and a lack of distinction when classifying forested areas at various scales. In this study, we introduce the DASR-Net algorithm, which integrates a dual attention network (DAN) in parallel with the Residual Network (ResNet) to enhance land cover classification, specifically focusing on improving the classification of forested regions. The dual attention mechanism within DASR-Net is designed to address the complexities inherent in forested landscapes by effectively capturing multiscale semantic information. This is achieved through multiscale null attention, which allows for the detailed examination of forest structures across different scales, and channel attention, which assigns weights to each channel to enhance feature expression using an improved BSE-ResNet bilinear approach. The two-channel parallel architecture of DASR-Net is particularly adept at resolving structural differences within forested areas, thereby avoiding information loss and the excessive fusion of features that can occur with traditional methods. This results in a more discriminative classification of remote sensing imagery, which is essential for accurate forest monitoring and management. To assess the efficacy of DASR-Net, we carried out tests with 10m Sentinel-2 multispectral remote sensing images over the Heshan District, which is renowned for its varied forestry. The findings reveal that the DASR-Net algorithm attains an accuracy rate of 96.36%, outperforming classical neural network models and the transformer (ViT) model. This demonstrates the scientific robustness and promise of the DASR-Net model in assisting with automatic object recognition for precise forest classification. Furthermore, we emphasize the relevance of our proposed model to hyperspectral datasets, which are frequently utilized in agricultural and forest classification tasks. DASR-Net’s enhanced feature extraction and classification capabilities are particularly advantageous for hyperspectral data, where the rich spectral information can be effectively harnessed to differentiate between various forest types and conditions. By doing so, DASR-Net contributes to advancing remote sensing applications in forest monitoring, supporting sustainable forestry practices and environmental conservation efforts. The findings of this study have significant practical implications for urban forestry management. The DASR-Net algorithm can enhance the accuracy of forest cover classification, aiding urban planners in better understanding and monitoring the status of urban forests. This, in turn, facilitates the development of effective forest conservation and restoration strategies, promoting the sustainable development of the urban ecological environment.
... Since its inception, U-Net has been adapted and extended for various applications. Variants like 3D U-Net [12], Attention U-Net [13], U-Net++ [14], nnUNet [15], and Conditional U-Net [16] have been developed to address specific challenges in medical imaging and beyond [17]. Recent research has focused on integrating FL with U-Net and its variants to improve privacy in medical image segmentation. ...
Brain tumor segmentation plays a crucial role in diagnosis and treatment planning. However, sharing patient data for training deep learning models raises privacy concerns. This paper proposes a federated learning (FL) approach utilizing a U-Net architecture for the segmentation of brain tumors. We evaluate the performance of federated U-Net models under various data distribution and varying numbers of clients. Specifically, we compare the effectiveness of two FL methods: Federated Averaging (FedAvg) and Federated Stochastic Gradient Descent (FedSGD). Through experiments conducted on the BraTS dataset, we observe that as the number of clients increases, the overall performance of the models tends to decrease. Moreover, we find that skewed data distribution often outperforms equal data division. Additionally, we consistently observe that FedAvg yields superior results compared to FedSGD. Our proposed approach enables hospitals to train models on local data collaboratively without directly sharing sensitive information. This preserves patient privacy while ensuring accurate tumor segmentation. The results of our study underscore the significance of strategic data distribution in FL environments and provide valuable insights for optimizing FL strategies in medical imaging applications.
