No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Lights and pitfalls of convolutional neural networks for diatom identification
... Deep-learning-based methods can achieve automatic feature learning through advanced network structures and complex features compared to simple ones. Therefore, with the development of deep learning technologies, increasing research about EM detection using deep learning methods is presented, such as in [81][82][83][84][85][86][87][88][89][90]. In [81][82][83], CNN was employed for EM detection. ...
... In [83], a CNN-based method was proposed for actinobacterial species detection. In [88], a region convolutional neural network (R-CNN)-based detector was proposed for diatom detection. In addition, a you only look once (YOLO)-based detector is prepared for comparisons. ...
... Based on our sufficient research foundation [15,[20][21][22]24,29], it can be found that only a few studies employed deep learning methods to perform the detection task in microorganism image analysis. Since the detection task in microorganism image analysis has strong application background, almost all studies directly utilize existing deep learning models, such as RCNN [88] and Faster R-CNN [84,85]. Different from these studies, we proposed a novel self-attention-based two-stage detection framework, which is inspired by ResNet, SENet, and FPN. ...
This paper proposes a novel Squeeze-and-excitation-based Mask Region Convolutional Neural Network (SEM-RCNN) for Environmental Microorganisms (EM) detection tasks. Mask RCNN, one of the most applied object detection models, uses ResNet for feature extraction. However, ResNet cannot combine the features of different image channels. To further optimize the feature extraction ability of the network, SEM-RCNN is proposed to combine the different features extracted by SENet and ResNet. The addition of SENet can allocate weight information when extracting features and increase the proportion of useful information. SEM-RCNN achieves a mean average precision (mAP) of 0.511 on EMDS-6. We further apply SEM-RCNN for blood-cell detection tasks on an open-source database (more than 17,000 microscopic images of blood cells) to verify the robustness and transferability of the proposed model. By comparing with other detectors based on deep learning, we demonstrate the superiority of SEM-RCNN in EM detection tasks. All experimental results show that the proposed SEM-RCNN exhibits excellent performances in EM detection.
... Result shows that the best validation accuracy is 96%. In Pedraza et al. (2018), the performance of R-CNN and you only look once (YOLO) in diatom detection is compared. The R-CNN based method consists of four fundamental steps as following, generating edge boxes for region proposal, rejecting proposed regions, classification of regions and box merging. ...
... Figure 43 shows deep learning models used in microorganism detection and the publication year of each model. As one of the classical deep learning models, CNN and its derivative models are often used in the task of microorganism detection, such as CNN mentioned in Panicker et al. (2018), Tahir et al. (2018), andSajedi et al. (2019), R-CNN mentioned in Pedraza et al. (2018), Faster R-CNN mentioned in Hung and Carpenter (2017), Viet et al. (2019), Baek et al. (2020), and Qian Pedraza et al. (2018) and Salido et al. (2020), Mask R-CNN mentioned in Ruiz-Santaquiteria et al. (2020). Among them, Faster R-CNN is the most commonly used method. ...
... Figure 43 shows deep learning models used in microorganism detection and the publication year of each model. As one of the classical deep learning models, CNN and its derivative models are often used in the task of microorganism detection, such as CNN mentioned in Panicker et al. (2018), Tahir et al. (2018), andSajedi et al. (2019), R-CNN mentioned in Pedraza et al. (2018), Faster R-CNN mentioned in Hung and Carpenter (2017), Viet et al. (2019), Baek et al. (2020), and Qian Pedraza et al. (2018) and Salido et al. (2020), Mask R-CNN mentioned in Ruiz-Santaquiteria et al. (2020). Among them, Faster R-CNN is the most commonly used method. ...
Microorganisms play a vital role in human life. Therefore, microorganism detection is of great significance to human beings. However, the traditional manual microscopic detection methods have the disadvantages of long detection cycle, low detection accuracy in large orders, and great difficulty in detecting uncommon microorganisms. Therefore, it is meaningful to apply computer image analysis technology to the field of microorganism detection. Computer image analysis can realize high-precision and high-efficiency detection of microorganisms. In this review, first,we analyse the existing microorganism detection methods in chronological order, from traditional image processing and traditional machine learning to deep learning methods. Then, we analyze and summarize these existing methods and introduce some potential methods, including visual transformers. In the end, the future development direction and challenges of microorganism detection are discussed. In general, we have summarized 142 related technical papers from 1985 to the present. This review will help researchers have a more comprehensive understanding of the development process, research status, and future trends in the field of microorganism detection and provide a reference for researchers in other fields.
... AlexNet is used to recognize 80 types of the diatom objects in the simple backgrounds [14]. They also applied the R-CNN [7] and the YOLO [16] deep learning network to detect the diatoms [15]. In addition, Zhou [31] et al. made a study on the use of GoogLeNet Inception V3 architecture for automatic diatom recognition, in which they only judged whether there were diatom objects in the image or not and counted how many diatoms there were in the image. ...
... In the third computer numerical simulation, the performance of the diatom recognition and the localization in the forensic investigation [15] via the applied network are validated in this paper. We point out that the values of the accuracy based on the deep learning methods are significantly higher than that of the traditional machine learning algorithms. ...
... Therefore, the feature extraction is no longer a bottleneck for improving the accuracy of the object recognition via the deep learning network. In addition, through the comparison and the demonstration, our applied deep network outperforms the existing work [15] from two aspects of viewpoints. 1) The recognition accuracy of the applied deep network is much higher than that of existing work [15]. It is because the ROIs are generated using the convoluted feature maps in our network. ...
In the forensic investigation, recognizing and locating the multiple diatom objects in an image is a challenging issue due to the interferences of the highly complex backgrounds. To address this issue, a coarse to fine diatom recognition and a localization framework based on the deep learning network is proposed in this paper. Firstly, the diatom images are obtained by performing the anatomic study on the cadavers. Next, a high definition electron microscope is scanned. Then, a coarse to fine deep learning framework is constructed to locate and recognize the diatom objects. Unlike the existing diatom classification and recognition methods, which used light microscopy with low resolution and completed under a simple backgrounds, our framework utilizes the high definition electron scanning microscopy with much higher resolution and suffers from the complex backgrounds interferences. To demonstrate the effectiveness of the proposed framework, 4 diatom image datasets with different background interference degrees are constructed. Also, 3 computer numerical simulation analysis are performed. They are (1) the limitations of the traditional methods in the diatom recognition, (2) the optimized composition of the training strategies and the network models, and (3) the performance of the proposed framework. The computer numerical simulation results show that the proposed framework achieves a recognition accuracy of 0.852. This is greater than 0.758 achieved by the AlexNet. Moreover, it can overcome the problem of the highly complex backgrounds interferences in the forensic investigation. Furthermore, it can locate and recognize the multiple objects in various diatom images simultaneously.
... For this reason, a great effort has recently been directed to developing faster and less cumbersome identification methods and metrics. These are mainly based either on DNA metabarcoding or a combination of diatom imaging acquisition and deep learning methods [13][14][15][16][17][18][19][20][21][22][23][24][25][26][27]. While these alternatives show promising results and take advantage of state-of-the-art sequencing and imaging methods, they are still laborious and quite expensive, which limits their application to routine monitoring of water quality. ...
... Compared to the available literature using ANN for automatic identification of diatoms, the use of Raman data required a remarkably smaller number of samples (spectra in our case). A previous study achieved an excellent accuracy (99.5%) using a total of 160,000 image samples processed by ANN to identify 80 diatom species (2000 samples per species) using a base dataset of 11,000 diatom samples [20]. Libreros and colleagues employed 16,000 segments of 365 images, combined with ANN, to identify diatom genera, achieving a classification accuracy of 74% [57]. ...
(1) An approach with great potential for fast and cost-effective profiling and identification of diatoms in lake ecosystems is presented herein. This approach takes advantage of Raman spectroscopy. (2) The study was based on the analysis of 790 Raman spectra from 29 species, belonging to 15 genera, 12 families, 9 orders and 4 subclasses, which were analysed using chemometric methods. The Raman data were first analysed by a partial least squares regression discriminant analysis (PLS-DA) to characterise the diatom species. Furthermore, a method was developed to streamline the integrated interpretation of PLS-DA when a high number of significant components is extracted. Subsequently, an artificial neural network (ANN) was used for taxa identification from Raman data. (3) The PLS interpretation produced a Raman profile for each species reflecting its biochemical composition. The ANN models were useful to identify various taxa with high accuracy. (4) Compared to studies in the literature, involving huge datasets one to four orders of magnitude larger than ours, high sensitivity was found for the identification of Achnanthidium exiguum (67%), Fragilaria pararumpens (67%), Amphora pediculus (71%), Achnanthidium minutissimum (80%) and Melosira varians (82%).
... Shi Z employed an improved YOLOV2 (You Only Look Once V2) model to detect the zooplankton in the holographic image data [17]. Pedraza et al. used CNN for the first time in automatic diatom classification and compared the performance between two state-of-the-art models RCNN (Region CNN) and YOLO [18,19]. Kerr T et al. proposed collaborative deep learning models to detect plankton from collected FlowCam image data to solve the problem of class imbalance [20]. ...
... Usually, there are two ways to improve the accuracy of the plankton detection and classification, one is to enrich the amount of the features and the other is to optimize the detection and classification model to reduce the feature loss. Most studies focus on augmenting the volume of the training dataset by rotating the original image data, changing the brightness and other operations [13][14][15][16][17][18][19][20]. Cheng et al. enriched the features of plankton by combining the features under both Cartesian and Polar coordinate systems, and then employed CNN and support vector machines (SVMs) to train the classification model and classify the taxa of plankton [14]. ...
Detecting and classifying the plankton in situ to analyze the population diversity and abundance is fundamental for the understanding of marine planktonic ecosystem. However, the features of plankton are subtle, and the distribution of different plankton taxa is extremely imbalanced in the real marine environment, both of which limit the detection and classification performance of them while implementing the advanced recognition models, especially for the rare taxa. In this paper, a novel plankton detection strategy is proposed combining with a cycle-consistent adversarial network and a densely connected YOLOV3 model, which not only solves the class imbalanced distribution problem of plankton by augmenting data volume for the rare taxa but also reduces the loss of the features in the plankton detection neural network. The mAP of the proposed plankton detection strategy achieved 97.21% and 97.14%, respectively, under two experimental datasets with a difference in the number of rare taxa, which demonstrated the superior performance of plankton detection comparing with other state-of-the-art models. Especially for the rare taxa, the detection accuracy for each rare taxa is improved by about 4.02% on average under the two experimental datasets. Furthermore, the proposed strategy may have the potential to be deployed into an autonomous underwater vehicle for mobile plankton ecosystem observation.
... Methods of segmentation applied to diatom detection in microscopic pictures are based primarily on traditional techniques such as regional segmentation [1,2], filtering [3,4], and deformable models [5][6][7]. There are currently only three works released using deep neural networks for diatom segmentation [8][9][10]. ...
... Comparison of R-CNN and YOLO was presented in [9] obtaining a better result for YOLO. Another comparison between classic and deep learning methods has been made in [41] and [10]. ...
This chapter presents different image segmentation methods that have been proven to be suitable for diatom detection. Methods are divided into classical and deep learning techniques. Moreover, within these methods, a distinction can be made according to the segmentation of images containing several taxon shells or single taxa. For this purpose, an overview of the most important contributions to diatom segmentation is performed. This survey covers a wide range of techniques from region and contour based on those using textural and frequency-based features and the state-of-the-art methods based on deep learning, such as semantic and instance segmentation.
... The publications included in this SLR for micro algae classification or detection mainly uses supervised machine learning technique, this is due to having data that is labeled and suitable for supervised techniques [75]. The deep learning network that is employed for algae classification or detection is convolutional neural network (CNN), some publications represent a framework based on multiple CNN and deep learning architectures as in [32], [34], [37], [46], [53], [54], [57], [61], [60], [62], and [63]. While some publications used traditional machine learning techniques for the classification or the detection of algae as in [23], [28] , [29] , [33], [40], [42], [59], and [61]. ...
Algae represent the majority of the diversity on Earth and are a large group of organisms that have photosynthetic properties that are important to life. The species of algae are estimated to be more than 1 million, they play an important role in many fields such as agriculture, industry, food, medicine. It is important to determine the type of algae to determine if it is harmful or useful, and to indicate the health of the ecosystem, water quality, health, and safety risks. The conventional process of classifying algae is difficult, tedious, and time-consuming. Recently various computer vision techniques have been used to classify algae to overcome challenges and automate the process of classification. This paper presents a review of research done on image classification for microorganism algae using machine learning and deep learning techniques. The paper focuses on three important research questions to highlight the challenges of classifying microalgae. A systematic literature review or SLR has been conducted to determine how deep learning and machine learning have improved and enhanced automatic microalgae classification rather than manual classification. 51 articles have been included from well-known databases. The outcome of this SLR is beneficial due to the detailed analysis and comprehensive overview of the algorithms and the architectures and information about the dataset used in each included article. The future work focuses on getting a large dataset with high resolution, trying different methods to manage imbalance problems, giving more attention to the fusion of deep learning techniques and traditional machine learning techniques.
... Modern CNN-based object detection methods such as R-CNN (Girshick et al., 2014), YOLO (Redmon et al., 2016), and their modifications perform the detection and recognition simultaneously, providing end-to-end methods for plankton recognition. For example, Pedraza et al. (2018) applied R-CNN to detect and classify diatoms in microscopy images, and Soh et al. (2018) used YOLO to detect and recognize plankton. Wang et al. (2022b) compared multiple CNN-based object detection methods including Faster R-CNN (Ren et al., 2017), SSD (Liu et al., 2016), YOLOv3 (Redmon and Farhadi, 2018) and YOLOX (Ge et al., 2021) on imaging flow cytometer data. ...
Planktonic organisms are key components of aquatic ecosystems and respond quickly to changes in the environment, therefore their monitoring is vital to understand the changes in the environment. Yet, monitoring plankton at appropriate scales still remains a challenge, limiting our understanding of functioning of aquatic systems and their response to changes. Modern plankton imaging instruments can be utilized to sample at high frequencies, enabling novel possibilities to study plankton populations. However, manual analysis of the data is costly, time consuming and expert based, making such approach unsuitable for large-scale application and urging for automatic solutions. The key problem related to the utilization of plankton datasets through image analysis is plankton recognition. Despite the large amount of research done, automatic methods have not been widely adopted for operational use. In this paper, a comprehensive survey on existing solutions for automatic plankton recognition is presented. First, we identify the most notable challenges that that make the development of plankton recognition systems difficult. Then, we provide a detailed description of solutions for these challenges proposed in plankton recognition literature. Finally, we propose a workflow to identify the specific challenges in new datasets and the recommended approaches to address them. For many of the challenges, applicable solutions exist. However, important challenges remain unsolved: 1) the domain shift between the datasets hindering the development of a general plankton recognition system that would work across different imaging instruments, 2) the difficulty to identify and process the images of previously unseen classes, and 3) the uncertainty in expert annotations that affects the training of the machine learning models for recognition. These challenges should be addressed in the future research.
... In addition, deep learning on image analysis is mainly focused on the identification of diatoms and algae, which would provide a potential strategy for the diagnosis of drowning. In 2018, Bueno et al. (2018) compared the functions of R-CNN and you only look once (YOLO) in diatom detection; the Fmeasure of YOLO was 84%. In 2019, Huang et al. applied and trained a CNN model based on the GoogLeNet Inception V3 architecture to identify diatoms with a validation rate of 97.33% (Zhou et al., 2019(Zhou et al., , 2020, which indicates that DL is an efficient and low-cost automated diatom detection technology. ...
Forensic microbiology has been widely used in the diagnosis of causes and manner of death, identification of individuals, detection of crime locations, and estimation of postmortem interval. However, the traditional method, microbial culture, has low efficiency, high consumption, and a low degree of quantitative analysis. With the development of high-throughput sequencing technology, advanced bioinformatics, and fast-evolving artificial intelligence, numerous machine learning models, such as RF, SVM, ANN, DNN, regression, PLS, ANOSIM, and ANOVA, have been established with the advancement of the microbiome and metagenomic studies. Recently, deep learning models, including the convolutional neural network (CNN) model and CNN-derived models, improve the accuracy of forensic prognosis using object detection techniques in microorganism image analysis. This review summarizes the application and development of forensic microbiology, as well as the research progress of machine learning (ML) and deep learning (DL) based on microbial genome sequencing and microbial images, and provided a future outlook on forensic microbiology.
... The adsorption of conditioning films on coatings is typically investigated in vitro for example by surface plasmon resonance spectroscopy with model biomacromolecules (Pranzetti et al. 2012;Koc et al. 2019) and unavoidable in natural living systems. For microfouling or "slime", there exists numerous automated microscopy based solutions with high accuracies for the detection or classification of foremost diatoms in drinking water samples (Coltelli et al. 2014;Pedraza et al. 2018;Tang et al. 2018;Ruiz-Santaquiteria et al. 2020) and in complex, mixed species environments (Deng et al. 2021) particularly on coatings after field tests (Krause et al. 2020). But for macrofouling, encountering the wide range of size scales and the differences between early and later stages of fouling is necessary. ...
Biofouling is a major challenge for sustainable shipping, filter membranes, heat exchangers, and medical devices. The development of fouling-resistant coatings requires the evaluation of their effectiveness. Such an evaluation is usually based on the assessment of fouling progression after different exposure times to the target medium (e.g. salt water). The manual assessment of macrofouling requires expert knowledge about local fouling communities due to high variances in phenotypical appearance, has single-image sampling inaccuracies for certain species, and lacks spatial information. Here an approach for automatic image-based macrofouling analysis was presented. A dataset with dense labels prepared from field panel images was made and a convolutional network (adapted U-Net) for the semantic segmentation of different macrofouling classes was proposed. The establishment of macrofouling localization allows for the generation of a successional model which enables the determination of direct surface attachment and in-depth epibiotic studies.
... Pedraza et al. [64] worked on deep learning-based neural to check the diatom detection from water. The authors determined the diatom detection with two popular transfer learning techniques i.e. ...
In the developing world, parasites are responsible for causing several serious health problems, with relatively high infections in human beings. The traditional manual light microscopy process of parasite recognition remains the golden standard approach for the diagnosis of parasitic species, but this approach is time-consuming, highly tedious, and also difficult to maintain consistency but essential in parasitological classification for carrying out several experimental observations. Therefore, it is meaningful to apply deep learning to address these challenges. Convolution Neural Network and digital slide scanning show promising results that can revolutionize the clinical parasitology laboratory by automating the process of classification and detection of parasites. Image analysis using deep learning methods have the potential to achieve high efficiency and accuracy. For this review, we have conducted a thorough investigation in the field of image detection and classification of various parasites based on deep learning. Online databases and digital libraries such as ACM, IEEE, ScienceDirect, Springer, and Wiley Online Library were searched to identify sufficient related paper collections. After screening of 200 research papers, 70 of them met our filtering criteria, which became a part of this study. This paper presents a comprehensive review of existing parasite classification and detection methods and models in chronological order, from traditional machine learning based techniques to deep learning based techniques. In this review, we also demonstrate the summary of machine learning and deep learning methods along with dataset details, evaluation metrics, methods limitations, and future scope over the one decade. The majority of the technical publications from 2012 to the present have been examined and summarized. In addition, we have discussed the future directions and challenges of parasites classification and detection to help researchers in understanding the existing research gaps. Further, this review provides support to researchers who require an effective and comprehensive understanding of deep learning development techniques, research, and future trends in the field of parasites detection and classification.
... The adsorption of conditioning films on coatings is typically investigated in vitro for example by surface plasmon resonance spectroscopy with model biomacromolecules (Pranzetti et al. 2012;Koc et al. 2019) and unavoidable in natural living systems. For microfouling or "slime", there exists numerous automated microscopy based solutions with high accuracies for the detection or classification of foremost diatoms in drinking water samples (Coltelli et al. 2014;Pedraza et al. 2018;Tang et al. 2018;Ruiz-Santaquiteria et al. 2020) and in complex, mixed species environments (Deng et al. 2021) particularly on coatings after field tests (Krause et al. 2020). But for macrofouling, encountering the wide range of size scales and the differences between early and later stages of fouling is necessary. ...
Biofouling is a major challenge for sustainable shipping, filter membranes, heat exchangers, and medical devices. The development of fouling-resistant coatings requires the evaluation of their effectiveness. Such an evaluation is usually based on the assessment of fouling progression after different exposure times to the target medium (e.g., salt water). The manual assessment of macrofouling requires expert knowledge about local fouling communities due to high variances in phenotypical appearance, has single-image sampling inaccuracies for certain species, and lacks spatial information. Here we present an approach for automatic image-based macrofouling analysis. We created a dataset with dense labels prepared from field panel images and propose a convolutional network (adapted U-Net) for the semantic segmentation of different macrofouling classes. The establishment of macrofouling localization allows for the generation of a successional model which enables the determination of direct surface attachment and in-depth epibiotic studies.
... This task can be time-consuming and subject to multiple biases due to the quality of the instrument or the level of expertise in diatom taxonomy. Therefore, attempts to automate the process have been ongoing since the early 90s (Du Buf et al., 1999;Pedraza et al., 2017Pedraza et al., , 2018Kloster et al., 2020). ...
Benthic diatoms are unicellular microalgae that are routinely used as bioindicators for monitoring the ecological status of freshwater. Their identification using light microscopy is a time-consuming and labor-intensive task that could be automated using deep-learning. However, training such networks relies on the availability of labeled datasets, which are difficult to obtain for these organisms. Herein, we propose a method to generate synthetic microscopy images for training. We gathered individual objects, i.e. 9230 diatoms from publicly available taxonomic guides and 600 items of debris from available real images. We collated a comprehensive dataset of synthetic microscopy images including both diatoms and debris using seamless blending and a combination of parameters such as image scaling, rotation, overlap and diatom-debris ratio. We then performed sensitivity analysis of the impact of the synthetic data parameters for training state-of-the art networks for horizontal and rotated bounding box detection (YOLOv5). We first trained the networks using the synthetic dataset and fine-tuned it to several real image datasets. Using this approach, the performance of the detection network was improved by up to 25% for precision and 23% for recall at an Intersection-over-Union(IoU) threshold of 0.5. This method will be extended in the future for training segmentation and classification networks.
... This situation is not friendly for practice and is apt to cause high false negative/positive rates due to fatigue and decreased concentration. It is of particular interest for academic research to explore the capability of automatically detecting the diatoms and/or recognizing the genera of the diatoms on optical microscope images (Bueno et al., 2018;Zhou et al., 2019Zhou et al., , 2020Kloster et al., 2020;Krause et al., 2020) or the SEM images Yu et al., 2021). These studies are inspired by the development of artificial intelligence recently and especially the giant success of deep learning (LeCun et al., 2015) in image processing and analysis, such as image classification, object detection, and region-of-interest (ROI) segmentation, which then makes it possible to build our own intelligent diatom test solution. ...
The diatom test is a forensic technique that can provide supportive evidence in the diagnosis of drowning but requires the laborious observation and counting of diatoms using a microscopy with too much effort, and therefore it is promising to introduce artificial intelligence (AI) to make the test process automatic. In this article, we propose an artificial intelligence solution based on the YOLOv5 framework for the automatic detection and recognition of the diatom genera. To evaluate the performance of this AI solution in different scenarios, we collected five lab-grown diatom genera and samples of some organic tissues from drowning cases to investigate the potential upper/lower limits of the capability in detecting the diatoms and recognizing their genera. Based on the study of the article, a recall score of 0.95 together with the corresponding precision score of 0.9 were achieved on the samples of the five lab-grown diatom genera via cross-validation, and the accuracy of the evaluation in the cases of kidney and liver is above 0.85 based on the precision and recall scores, which demonstrate the effectiveness of the AI solution to be used in drowning forensic routine.
... The above algorithms have both been applied in object detection tasks for marine phytoplankton observation. For example, A. Pedraza et al [18] applied the object detection models RCNN and YOLO to 10 classes of diatom microscopic images. Y. Li et al. [19] designed a Dense-YOLOv3 network to improve the mAP of selected eight categories of phytoplankton on WHOI-Plankton dataset. ...
Rapid and quantitative analysis of phytoplankton cells in natural seawater is of great need for marine ecological science research and harmful algae bloom monitoring applications. In this paper, we propose a YOLOX network-based object detection algorithm exclusively for high-throughput real-time analysis of phytoplankton fluorescence images collected by the FluoSieve® imaging flow cytometer. Based on an active learning strategy, we first annotate and construct a FluoPhyto dataset of red tide phytoplankton species fluorescence images commonly found in the South and East China Sea, which contains a total of 30,339 images in 32 different categories. Using the dataset, we train the Faster-RCNN, SSD, YOLOv3 and YOLOX networks, and the comparison result shows that the performance of YOLOX network outperforms the other networks, which can reach a mean average precision (mAP) of 90.9%. The trained YOLOX model is then deployed on an embedded GPU module and the inference speed is tested to reach 20 fps with the help of TensorRT optimization, which can hopefully meet the real-time detection requirements of the instrument. In addition, the algorithm is run on the embedded platform for detection of images collected in a red tide event that happened near the Pearl River Estuary, and the key parameters such as abundance and size spectrum of the dominant species, Cochlodinium geminatum, are obtained, which confirms the feasibility and superior performance of the detection algorithm.
... The experimental results show that the accuracy of detection is 94.8%. In Pedraza et al. (2018), a comparison is conducted to test whether the latest deep learning network, including RCNN and YOLO, can adapt to the detection of diatoms. The data used here is private and contains nearly 11,000 images in ten categories. ...
Microorganisms are widely distributed in the human daily living environment. They play an essential role in environmental pollution control, disease prevention and treatment, and food and drug production. The analysis of microorganisms is essential for making full use of different microorganisms. The conventional analysis methods are laborious and time-consuming. Therefore, the automatic image analysis based on artificial neural networks is introduced to optimize it. However, the automatic microorganism image analysis faces many challenges, such as the requirement of a robust algorithm caused by various application occasions, insignificant features and easy under-segmentation caused by the image characteristic, and various analysis tasks. Therefore, we conduct this review to comprehensively discuss the characteristics of microorganism image analysis based on artificial neural networks. In this review, the background and motivation are introduced first. Then, the development of artificial neural networks and representative networks are presented. After that, the papers related to microorganism image analysis based on classical and deep neural networks are reviewed from the perspectives of different tasks. In the end, the methodology analysis and potential direction are discussed.
Background and objective:
Diatom testing is supportive for drowning diagnosis in forensic medicine. However, it is very time-consuming and labor-intensive for technicians to identify microscopically a handful of diatoms in sample smears, especially under complex observable backgrounds. Recently, we successfully developed a software, named DiatomNet v1.0 intended to automatically identify diatom frustules in a whole slide under a clear background. Here, we introduced this new software and performed a validation study to elucidate how DiatomNet v1.0 improved its performance with the influence of visible impurities.
Methods:
DiatomNet v1.0 has an intuitive, user-friendly and easy-to-learn graphical user interface (GUI) built in the Drupal and its core architecture for slide analysis including a convolutional neural network (CNN) is written in Python language. The build-in CNN model was evaluated for diatom identification under very complex observable backgrounds with mixtures of common impurities, including carbon pigments and sand sediments. Compared to the original model, the enhanced model following optimization with limited new datasets was evaluated systematically by independent testing and random control trials (RCTs).
Results:
In independent testing, the original DiatomNet v1.0 was moderately affected, especially when higher densities of impurities existed, and achieved a low recall of 0.817 and F1 score of 0.858 but good precision of 0.905. Following transfer learning with limited new datasets, the enhanced version had better results, with recall and F1 score values of 0.968. A comparative study on real slides showed that the upgraded DiatomNet v1.0 obtained F1 scores of 0.86 and 0.84 for carbon pigment and sand sediment, respectively, slightly worse than manual identification (carbon pigment: 0.91; sand sediment: 0.86), but much less time was needed.
Conclusions:
The study verified that forensic diatom testing with aid of DiatomNet v1.0 is much more efficient than traditionally manual identification even under complex observable backgrounds. In terms of forensic diatom testing, we proposed a suggested standard on build-in model optimization and evaluation to strengthen the software's generalization in potentially complex conditions.
In the recent era of AI, instance segmentation has significantly advanced boundary and object detection especially in diverse fields (e.g., biological and environmental research). Despite its progress, edge detection amid adjacent objects (e.g., organism cells) still remains intractable. This is because homogeneous and heterogeneous objects are prone to being mingled in a single image. To cope with this challenge, we propose the weighted Mask R-CNN designed to effectively separate overlapped objects in virtue of extra weights to adjacent boundaries. For numerical study, a range of experiments are performed with applications to simulated data and real data (e.g., Microcystis, one of the most common algae genera and cell membrane images). It is noticeable that the weighted Mask R-CNN outperforms the standard Mask R-CNN, given that the analytic experiments show on average 92.5% of precision and 96.4% of recall in algae data and 94.5% of precision and 98.6% of recall in cell membrane data. Consequently, we found that a majority of sample boundaries in real and simulated data are precisely segmented in the midst of object mixtures.
The Eucampia Index, which is calculated from valve ratio of Antarctic diatom Eucampia ainarctica varieties, has been expected to be a useful indicator of sea ice coverage or/and sea surface temperature variation in the Southern Ocean. To verify the relationship between the index value and the environmental factors, considerable effort is needed to classify and count valves of E. antarctica in a very large number of samples. In this study, to realize automated detection of the Eucampia Index, we constructed a deep-learning (one of the learning methods of artificial intelligence) based models for identifying Eucampia valves from various particles in a diatom slide. The microfossil Classification and Rapid Accumulation Device (miCRAD) system, which can be used for scanning a slide and cropping images of particles automatically, was employed to collect images in training dataset for the model and test dataset for model verification. As a result of classifying particle images in the test dataset by the initial model "Eant_1000px_200616", accuracy was 78.8%. The Eucampia Index value prepared in the test dataset was 0.80, and the value predicted using the developed model from the same dataset was 0.76. The predicted value was in the range of the manual counting error. These results suggest that the classification performance of the model is similar to that of a human expert. This study revealed that a model capable of detecting the ratio of two diatom species can be constructed using the miCRAD system for the first time. The miCRAD system connected with the developed model in this study is capable of automatically classifying particle images at the same time of capturing images so that the system can be applied to a large-scale analysis of the Eucampia index in the Southern Ocean. Depending on the setting of the classification category, similar method is relevant to investigators who have to process a large number of diatom samples such as for detecting specific species for biostratigraphic and paleoenvironmental studies.
Forensic diatom test has been widely accepted as a way of providing supportive evidences in the diagnosis of drowning. The current workflow is primarily based on the observation of diatoms by forensic pathologists under a microscopy, and this process can be very time-consuming. In this paper, we demonstrate a deep learning-based approach for automatically searching diatoms in scanning electron microscopic images. Cross-validation studies were performed to evaluate the influence of magnification on performance. Moreover, various training strategies were tested to improve the performance of detection. The conclusion shows that our approach can satisfy the necessary requirements to be integrated as part of an automatic forensic diatom test.
While the use of deep learning is a valuable technology for automatic detection systems for medical data and images, the biofouling community is still lacking an analytical tool for the detection and counting of diatoms on samples after short-term field exposure. In this work a fully convolutional neural network was implemented as a fast and simple approach to detect diatoms on two-channel (fluorescence and phase contrast) microscopy images by predicting bounding boxes. The developed approach performs well with only a small number of trainable parameters and a F1 score of 0.82. Counting diatoms was evaluated on a dataset of 600 microscopy images of three different surface chemistries (hydrophilic and hydrophobic) and is very similar to counting by humans, while demanding only a fraction of the analysis time.
Diatom identification is a crucial process to estimate water quality, which is essential in biological studies. This process can be automated with machine learning algorithms. For this purpose, a dataset with 10 common taxa is collected, with annotations provided by an expert diatomist. In this work, a comparison of the classical state-of-the-art general purpose methods along with two different deep learning approaches is carried out. The classical methods are based on Viola-Jones and scale and curvature invariant ridge object detectors. The deep learning based methods are Semantic Segmentation and YOLO. This is the first time that Viola-Jones and Semantic Segmentation techniques are applied and compared for diatom segmentation in microscopic images containing several taxon shells. While all methods provide relatively good results in specific species, the deep learning approaches are consistently better in terms of sensitivity and specificity (up to 0.99 for some taxa) and up to 0.86 precision.
This paper deals with automatic taxa identification based on machine learning methods. The aim is therefore to automatically classify diatoms, in terms of pattern recognition terminology. Diatoms are a kind of algae microorganism with high biodiversity at the species level, which are useful for water quality assessment. The most relevant features for diatom description and classification have been selected using an extensive dataset of 80 taxa with a minimum of 100 samples/taxon augmented to 300 samples/taxon. In addition to published morphological, statistical and textural descriptors, a new textural descriptor, Local Binary Patterns (LBP), to characterize the diatom’s valves, and a log Gabor implementation not tested before for this purpose are introduced in this paper. Results show an overall accuracy of 98.11% using bagging decision trees and combinations of descriptors. Finally, some phycological features of diatoms that are still difficult to integrate in computer systems are discussed for future work.
Diatoms, a kind of algae microorganisms with several species, are quite useful for water quality determination, one of the hottest topics in applied biology nowadays. At the same time, deep learning and convolutional neural networks (CNN) are becoming an extensively used technique for image classification in a variety of problems. This paper approaches diatom classification with this technique, in order to demonstrate whether it is suitable for solving the classification problem. An extensive dataset was specifically collected (80 types, 100 samples/type) for this study. The dataset covers different illumination conditions and it was computationally augmented to more than 160,000 samples. After that, CNNs were applied over datasets pre-processed with different image processing techniques. An overall accuracy of 99% is obtained for the 80-class problem and different kinds of images (brightfield, normalized). Results were compared to previous presented classification techniques with different number of samples. As far as the authors know, this is the first time that CNNs are applied to diatom classification.
Receiver operating characteristics (ROC) graphs are useful for organizing classifiers and visualizing their performance. ROC graphs are commonly used in medical decision making, and in recent years have been used increasingly in machine learning and data mining research. Although ROC graphs are apparently simple, there are some common misconceptions and pitfalls when using them in practice. The purpose of this article is to serve as an introduction to ROC graphs and as a guide for using them in research.
The use of object proposals is an effective recent approach for increasing the computational efficiency of object detection. We propose a novel method for generating object bounding box proposals using edges. Edges provide a sparse yet informative representation of an image. Our main observation is that the number of contours that are wholly contained in a bounding box is indicative of the likelihood of the box containing an object. We propose a simple box objectness score that measures the number of edges that exist in the box minus those that are members of contours that overlap the box's boundary. Using efficient data structures, millions of candidate boxes can be evaluated in a fraction of a second, returning a ranked set of a few thousand top-scoring proposals. Using standard metrics, we show results that are significantly more accurate than the current state-of-the-art while being faster to compute. In particular, given just 1000 proposals we achieve over 96% object recall at overlap threshold of 0.5 and over 75% recall at the more challenging overlap of 0.7. Our approach runs in 0.25 seconds and we additionally demonstrate a near real-time variant with only minor loss in accuracy.
We introduce YOLO9000, a state-of-the-art, real-time object detection system that can detect over 9000 object categories. First we propose various improvements to the YOLO detection method, both novel and drawn from prior work. The improved model, YOLOv2, is state-of-the-art on standard detection tasks like PASCAL VOC and COCO. At 67 FPS, YOLOv2 gets 76.8 mAP on VOC 2007. At 40 FPS, YOLOv2 gets 78.6 mAP, outperforming state-of-the-art methods like Faster RCNN with ResNet and SSD while still running significantly faster. Finally we propose a method to jointly train on object detection and classification. Using this method we train YOLO9000 simultaneously on the COCO detection dataset and the ImageNet classification dataset. Our joint training allows YOLO9000 to predict detections for object classes that don't have labelled detection data. We validate our approach on the ImageNet detection task. YOLO9000 gets 19.7 mAP on the ImageNet detection validation set despite only having detection data for 44 of the 200 classes. On the 156 classes not in COCO, YOLO9000 gets 16.0 mAP. But YOLO can detect more than just 200 classes; it predicts detections for more than 9000 different object categories. And it still runs in real-time.
Darknet: Open source neural networks in c.
- Redmon
Redmon, J., "Darknet: Open source neural networks in c." http://pjreddie.com/darknet/ (2013-2016).
(Accessed: 12 March 2018).