Conference Paper

Lights and pitfalls of convolutional neural networks for diatom identification

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Result shows that the best validation accuracy is 96%. In Pedraza et al. (2018), the performance of R-CNN and you only look once (YOLO) in diatom detection is compared. The R-CNN based method consists of four fundamental steps as following, generating edge boxes for region proposal, rejecting proposed regions, classification of regions and box merging. ...
... Figure 43 shows deep learning models used in microorganism detection and the publication year of each model. As one of the classical deep learning models, CNN and its derivative models are often used in the task of microorganism detection, such as CNN mentioned in Panicker et al. (2018), Tahir et al. (2018), andSajedi et al. (2019), R-CNN mentioned in Pedraza et al. (2018), Faster R-CNN mentioned in Hung and Carpenter (2017), Viet et al. (2019), Baek et al. (2020), and Qian Pedraza et al. (2018) and Salido et al. (2020), Mask R-CNN mentioned in Ruiz-Santaquiteria et al. (2020). Among them, Faster R-CNN is the most commonly used method. ...
... Figure 43 shows deep learning models used in microorganism detection and the publication year of each model. As one of the classical deep learning models, CNN and its derivative models are often used in the task of microorganism detection, such as CNN mentioned in Panicker et al. (2018), Tahir et al. (2018), andSajedi et al. (2019), R-CNN mentioned in Pedraza et al. (2018), Faster R-CNN mentioned in Hung and Carpenter (2017), Viet et al. (2019), Baek et al. (2020), and Qian Pedraza et al. (2018) and Salido et al. (2020), Mask R-CNN mentioned in Ruiz-Santaquiteria et al. (2020). Among them, Faster R-CNN is the most commonly used method. ...
Article
Full-text available
Microorganisms play a vital role in human life. Therefore, microorganism detection is of great significance to human beings. However, the traditional manual microscopic detection methods have the disadvantages of long detection cycle, low detection accuracy in large orders, and great difficulty in detecting uncommon microorganisms. Therefore, it is meaningful to apply computer image analysis technology to the field of microorganism detection. Computer image analysis can realize high-precision and high-efficiency detection of microorganisms. In this review, first,we analyse the existing microorganism detection methods in chronological order, from traditional image processing and traditional machine learning to deep learning methods. Then, we analyze and summarize these existing methods and introduce some potential methods, including visual transformers. In the end, the future development direction and challenges of microorganism detection are discussed. In general, we have summarized 142 related technical papers from 1985 to the present. This review will help researchers have a more comprehensive understanding of the development process, research status, and future trends in the field of microorganism detection and provide a reference for researchers in other fields.
... AlexNet is used to recognize 80 types of the diatom objects in the simple backgrounds [14]. They also applied the R-CNN [7] and the YOLO [16] deep learning network to detect the diatoms [15]. In addition, Zhou [31] et al. made a study on the use of GoogLeNet Inception V3 architecture for automatic diatom recognition, in which they only judged whether there were diatom objects in the image or not and counted how many diatoms there were in the image. ...
... In the third computer numerical simulation, the performance of the diatom recognition and the localization in the forensic investigation [15] via the applied network are validated in this paper. We point out that the values of the accuracy based on the deep learning methods are significantly higher than that of the traditional machine learning algorithms. ...
... Therefore, the feature extraction is no longer a bottleneck for improving the accuracy of the object recognition via the deep learning network. In addition, through the comparison and the demonstration, our applied deep network outperforms the existing work [15] from two aspects of viewpoints. 1) The recognition accuracy of the applied deep network is much higher than that of existing work [15]. It is because the ROIs are generated using the convoluted feature maps in our network. ...
Article
Full-text available
In the forensic investigation, recognizing and locating the multiple diatom objects in an image is a challenging issue due to the interferences of the highly complex backgrounds. To address this issue, a coarse to fine diatom recognition and a localization framework based on the deep learning network is proposed in this paper. Firstly, the diatom images are obtained by performing the anatomic study on the cadavers. Next, a high definition electron microscope is scanned. Then, a coarse to fine deep learning framework is constructed to locate and recognize the diatom objects. Unlike the existing diatom classification and recognition methods, which used light microscopy with low resolution and completed under a simple backgrounds, our framework utilizes the high definition electron scanning microscopy with much higher resolution and suffers from the complex backgrounds interferences. To demonstrate the effectiveness of the proposed framework, 4 diatom image datasets with different background interference degrees are constructed. Also, 3 computer numerical simulation analysis are performed. They are (1) the limitations of the traditional methods in the diatom recognition, (2) the optimized composition of the training strategies and the network models, and (3) the performance of the proposed framework. The computer numerical simulation results show that the proposed framework achieves a recognition accuracy of 0.852. This is greater than 0.758 achieved by the AlexNet. Moreover, it can overcome the problem of the highly complex backgrounds interferences in the forensic investigation. Furthermore, it can locate and recognize the multiple objects in various diatom images simultaneously.
... For this reason, a great effort has recently been directed to developing faster and less cumbersome identification methods and metrics. These are mainly based either on DNA metabarcoding or a combination of diatom imaging acquisition and deep learning methods [13][14][15][16][17][18][19][20][21][22][23][24][25][26][27]. While these alternatives show promising results and take advantage of state-of-the-art sequencing and imaging methods, they are still laborious and quite expensive, which limits their application to routine monitoring of water quality. ...
... Compared to the available literature using ANN for automatic identification of diatoms, the use of Raman data required a remarkably smaller number of samples (spectra in our case). A previous study achieved an excellent accuracy (99.5%) using a total of 160,000 image samples processed by ANN to identify 80 diatom species (2000 samples per species) using a base dataset of 11,000 diatom samples [20]. Libreros and colleagues employed 16,000 segments of 365 images, combined with ANN, to identify diatom genera, achieving a classification accuracy of 74% [57]. ...
Article
Full-text available
(1) An approach with great potential for fast and cost-effective profiling and identification of diatoms in lake ecosystems is presented herein. This approach takes advantage of Raman spectroscopy. (2) The study was based on the analysis of 790 Raman spectra from 29 species, belonging to 15 genera, 12 families, 9 orders and 4 subclasses, which were analysed using chemometric methods. The Raman data were first analysed by a partial least squares regression discriminant analysis (PLS-DA) to characterise the diatom species. Furthermore, a method was developed to streamline the integrated interpretation of PLS-DA when a high number of significant components is extracted. Subsequently, an artificial neural network (ANN) was used for taxa identification from Raman data. (3) The PLS interpretation produced a Raman profile for each species reflecting its biochemical composition. The ANN models were useful to identify various taxa with high accuracy. (4) Compared to studies in the literature, involving huge datasets one to four orders of magnitude larger than ours, high sensitivity was found for the identification of Achnanthidium exiguum (67%), Fragilaria pararumpens (67%), Amphora pediculus (71%), Achnanthidium minutissimum (80%) and Melosira varians (82%).
... Shi Z employed an improved YOLOV2 (You Only Look Once V2) model to detect the zooplankton in the holographic image data [17]. Pedraza et al. used CNN for the first time in automatic diatom classification and compared the performance between two state-of-the-art models RCNN (Region CNN) and YOLO [18,19]. Kerr T et al. proposed collaborative deep learning models to detect plankton from collected FlowCam image data to solve the problem of class imbalance [20]. ...
... Usually, there are two ways to improve the accuracy of the plankton detection and classification, one is to enrich the amount of the features and the other is to optimize the detection and classification model to reduce the feature loss. Most studies focus on augmenting the volume of the training dataset by rotating the original image data, changing the brightness and other operations [13][14][15][16][17][18][19][20]. Cheng et al. enriched the features of plankton by combining the features under both Cartesian and Polar coordinate systems, and then employed CNN and support vector machines (SVMs) to train the classification model and classify the taxa of plankton [14]. ...
Article
Full-text available
Detecting and classifying the plankton in situ to analyze the population diversity and abundance is fundamental for the understanding of marine planktonic ecosystem. However, the features of plankton are subtle, and the distribution of different plankton taxa is extremely imbalanced in the real marine environment, both of which limit the detection and classification performance of them while implementing the advanced recognition models, especially for the rare taxa. In this paper, a novel plankton detection strategy is proposed combining with a cycle-consistent adversarial network and a densely connected YOLOV3 model, which not only solves the class imbalanced distribution problem of plankton by augmenting data volume for the rare taxa but also reduces the loss of the features in the plankton detection neural network. The mAP of the proposed plankton detection strategy achieved 97.21% and 97.14%, respectively, under two experimental datasets with a difference in the number of rare taxa, which demonstrated the superior performance of plankton detection comparing with other state-of-the-art models. Especially for the rare taxa, the detection accuracy for each rare taxa is improved by about 4.02% on average under the two experimental datasets. Furthermore, the proposed strategy may have the potential to be deployed into an autonomous underwater vehicle for mobile plankton ecosystem observation.
... Methods of segmentation applied to diatom detection in microscopic pictures are based primarily on traditional techniques such as regional segmentation [1,2], filtering [3,4], and deformable models [5][6][7]. There are currently only three works released using deep neural networks for diatom segmentation [8][9][10]. ...
... Comparison of R-CNN and YOLO was presented in [9] obtaining a better result for YOLO. Another comparison between classic and deep learning methods has been made in [41] and [10]. ...
Chapter
This chapter presents different image segmentation methods that have been proven to be suitable for diatom detection. Methods are divided into classical and deep learning techniques. Moreover, within these methods, a distinction can be made according to the segmentation of images containing several taxon shells or single taxa. For this purpose, an overview of the most important contributions to diatom segmentation is performed. This survey covers a wide range of techniques from region and contour based on those using textural and frequency-based features and the state-of-the-art methods based on deep learning, such as semantic and instance segmentation.
... The experimental results show that the accuracy of detection is 94.8%. In Pedraza et al. (2018), a comparison is conducted to test whether the latest deep learning network, including RCNN and YOLO, can adapt to the detection of diatoms. The data used here is private and contains nearly 11,000 images in ten categories. ...
Article
Full-text available
Microorganisms are widely distributed in the human daily living environment. They play an essential role in environmental pollution control, disease prevention and treatment, and food and drug production. The analysis of microorganisms is essential for making full use of different microorganisms. The conventional analysis methods are laborious and time-consuming. Therefore, the automatic image analysis based on artificial neural networks is introduced to optimize it. However, the automatic microorganism image analysis faces many challenges, such as the requirement of a robust algorithm caused by various application occasions, insignificant features and easy under-segmentation caused by the image characteristic, and various analysis tasks. Therefore, we conduct this review to comprehensively discuss the characteristics of microorganism image analysis based on artificial neural networks. In this review, the background and motivation are introduced first. Then, the development of artificial neural networks and representative networks are presented. After that, the papers related to microorganism image analysis based on classical and deep neural networks are reviewed from the perspectives of different tasks. In the end, the methodology analysis and potential direction are discussed.
... The above algorithms have both been applied in object detection tasks for marine phytoplankton observation. For example, A. Pedraza et al [18] applied the object detection models RCNN and YOLO to 10 classes of diatom microscopic images. Y. Li et al. [19] designed a Dense-YOLOv3 network to improve the mAP of selected eight categories of phytoplankton on WHOI-Plankton dataset. ...
Preprint
Full-text available
Rapid and quantitative analysis of phytoplankton cells in natural seawater is of great need for marine ecological science research and harmful algae bloom monitoring applications. In this paper, we propose a YOLOX network-based object detection algorithm exclusively for high-throughput real-time analysis of phytoplankton fluorescence images collected by the FluoSieve ® imaging flow cytometer. Based on an active learning strategy, we first annotate and construct a FluoPhyto dataset of red tide phytoplankton species fluorescence images commonly found in the South and East China Sea, which contains a total of 30,339 images in 32 different categories. Using the dataset, we train the Faster-RCNN, SSD, YOLOv3 and YOLOX networks, and the comparison result shows that the performance of YOLOX network outperforms the other networks, which can reach a mean average precision (mAP) of 90.9%. The trained YOLOX model is then deployed on an embedded GPU module and the inference speed is tested to reach 20 fps with the help of TensorRT optimization, which can hopefully meet the real-time detection requirements of the instrument. In addition, the algorithm is run on the embedded platform for detection of images collected in a red tide event that happened near the Pearl River Estuary, and the key parameters such as abundance and size spectrum of the dominant species, Cochlodinium geminatum, are obtained, which confirms the feasibility and superior performance of the detection algorithm.
Article
Full-text available
In the recent era of AI, instance segmentation has significantly advanced boundary and object detection especially in diverse fields (e.g., biological and environmental research). Despite its progress, edge detection amid adjacent objects (e.g., organism cells) still remains intractable. This is because homogeneous and heterogeneous objects are prone to being mingled in a single image. To cope with this challenge, we propose the weighted Mask R-CNN designed to effectively separate overlapped objects in virtue of extra weights to adjacent boundaries. For numerical study, a range of experiments are performed with applications to simulated data and real data (e.g., Microcystis, one of the most common algae genera and cell membrane images). It is noticeable that the weighted Mask R-CNN outperforms the standard Mask R-CNN, given that the analytic experiments show on average 92.5% of precision and 96.4% of recall in algae data and 94.5% of precision and 98.6% of recall in cell membrane data. Consequently, we found that a majority of sample boundaries in real and simulated data are precisely segmented in the midst of object mixtures.
Preprint
Full-text available
The Eucampia Index, which is calculated from valve ratio of Antarctic diatom Eucampia ainarctica varieties, has been expected to be a useful indicator of sea ice coverage or/and sea surface temperature variation in the Southern Ocean. To verify the relationship between the index value and the environmental factors, considerable effort is needed to classify and count valves of E. antarctica in a very large number of samples. In this study, to realize automated detection of the Eucampia Index, we constructed a deep-learning (one of the learning methods of artificial intelligence) based models for identifying Eucampia valves from various particles in a diatom slide. The microfossil Classification and Rapid Accumulation Device (miCRAD) system, which can be used for scanning a slide and cropping images of particles automatically, was employed to collect images in training dataset for the model and test dataset for model verification. As a result of classifying particle images in the test dataset by the initial model "Eant_1000px_200616", accuracy was 78.8%. The Eucampia Index value prepared in the test dataset was 0.80, and the value predicted using the developed model from the same dataset was 0.76. The predicted value was in the range of the manual counting error. These results suggest that the classification performance of the model is similar to that of a human expert. This study revealed that a model capable of detecting the ratio of two diatom species can be constructed using the miCRAD system for the first time. The miCRAD system connected with the developed model in this study is capable of automatically classifying particle images at the same time of capturing images so that the system can be applied to a large-scale analysis of the Eucampia index in the Southern Ocean. Depending on the setting of the classification category, similar method is relevant to investigators who have to process a large number of diatom samples such as for detecting specific species for biostratigraphic and paleoenvironmental studies.
Article
Full-text available
Forensic diatom test has been widely accepted as a way of providing supportive evidences in the diagnosis of drowning. The current workflow is primarily based on the observation of diatoms by forensic pathologists under a microscopy, and this process can be very time-consuming. In this paper, we demonstrate a deep learning-based approach for automatically searching diatoms in scanning electron microscopic images. Cross-validation studies were performed to evaluate the influence of magnification on performance. Moreover, various training strategies were tested to improve the performance of detection. The conclusion shows that our approach can satisfy the necessary requirements to be integrated as part of an automatic forensic diatom test.
Article
While the use of deep learning is a valuable technology for automatic detection systems for medical data and images, the biofouling community is still lacking an analytical tool for the detection and counting of diatoms on samples after short-term field exposure. In this work a fully convolutional neural network was implemented as a fast and simple approach to detect diatoms on two-channel (fluorescence and phase contrast) microscopy images by predicting bounding boxes. The developed approach performs well with only a small number of trainable parameters and a F1 score of 0.82. Counting diatoms was evaluated on a dataset of 600 microscopy images of three different surface chemistries (hydrophilic and hydrophobic) and is very similar to counting by humans, while demanding only a fraction of the analysis time.
Conference Paper
Diatom identification is a crucial process to estimate water quality, which is essential in biological studies. This process can be automated with machine learning algorithms. For this purpose, a dataset with 10 common taxa is collected, with annotations provided by an expert diatomist. In this work, a comparison of the classical state-of-the-art general purpose methods along with two different deep learning approaches is carried out. The classical methods are based on Viola-Jones and scale and curvature invariant ridge object detectors. The deep learning based methods are Semantic Segmentation and YOLO. This is the first time that Viola-Jones and Semantic Segmentation techniques are applied and compared for diatom segmentation in microscopic images containing several taxon shells. While all methods provide relatively good results in specific species, the deep learning approaches are consistently better in terms of sensitivity and specificity (up to 0.99 for some taxa) and up to 0.86 precision.
Article
Full-text available
This paper deals with automatic taxa identification based on machine learning methods. The aim is therefore to automatically classify diatoms, in terms of pattern recognition terminology. Diatoms are a kind of algae microorganism with high biodiversity at the species level, which are useful for water quality assessment. The most relevant features for diatom description and classification have been selected using an extensive dataset of 80 taxa with a minimum of 100 samples/taxon augmented to 300 samples/taxon. In addition to published morphological, statistical and textural descriptors, a new textural descriptor, Local Binary Patterns (LBP), to characterize the diatom’s valves, and a log Gabor implementation not tested before for this purpose are introduced in this paper. Results show an overall accuracy of 98.11% using bagging decision trees and combinations of descriptors. Finally, some phycological features of diatoms that are still difficult to integrate in computer systems are discussed for future work.
Article
Full-text available
Diatoms, a kind of algae microorganisms with several species, are quite useful for water quality determination, one of the hottest topics in applied biology nowadays. At the same time, deep learning and convolutional neural networks (CNN) are becoming an extensively used technique for image classification in a variety of problems. This paper approaches diatom classification with this technique, in order to demonstrate whether it is suitable for solving the classification problem. An extensive dataset was specifically collected (80 types, 100 samples/type) for this study. The dataset covers different illumination conditions and it was computationally augmented to more than 160,000 samples. After that, CNNs were applied over datasets pre-processed with different image processing techniques. An overall accuracy of 99% is obtained for the 80-class problem and different kinds of images (brightfield, normalized). Results were compared to previous presented classification techniques with different number of samples. As far as the authors know, this is the first time that CNNs are applied to diatom classification.
Article
Full-text available
Receiver operating characteristics (ROC) graphs are useful for organizing classifiers and visualizing their performance. ROC graphs are commonly used in medical decision making, and in recent years have been used increasingly in machine learning and data mining research. Although ROC graphs are apparently simple, there are some common misconceptions and pitfalls when using them in practice. The purpose of this article is to serve as an introduction to ROC graphs and as a guide for using them in research.
Conference Paper
The use of object proposals is an effective recent approach for increasing the computational efficiency of object detection. We propose a novel method for generating object bounding box proposals using edges. Edges provide a sparse yet informative representation of an image. Our main observation is that the number of contours that are wholly contained in a bounding box is indicative of the likelihood of the box containing an object. We propose a simple box objectness score that measures the number of edges that exist in the box minus those that are members of contours that overlap the box's boundary. Using efficient data structures, millions of candidate boxes can be evaluated in a fraction of a second, returning a ranked set of a few thousand top-scoring proposals. Using standard metrics, we show results that are significantly more accurate than the current state-of-the-art while being faster to compute. In particular, given just 1000 proposals we achieve over 96% object recall at overlap threshold of 0.5 and over 75% recall at the more challenging overlap of 0.7. Our approach runs in 0.25 seconds and we additionally demonstrate a near real-time variant with only minor loss in accuracy.
Article
We introduce YOLO9000, a state-of-the-art, real-time object detection system that can detect over 9000 object categories. First we propose various improvements to the YOLO detection method, both novel and drawn from prior work. The improved model, YOLOv2, is state-of-the-art on standard detection tasks like PASCAL VOC and COCO. At 67 FPS, YOLOv2 gets 76.8 mAP on VOC 2007. At 40 FPS, YOLOv2 gets 78.6 mAP, outperforming state-of-the-art methods like Faster RCNN with ResNet and SSD while still running significantly faster. Finally we propose a method to jointly train on object detection and classification. Using this method we train YOLO9000 simultaneously on the COCO detection dataset and the ImageNet classification dataset. Our joint training allows YOLO9000 to predict detections for object classes that don't have labelled detection data. We validate our approach on the ImageNet detection task. YOLO9000 gets 19.7 mAP on the ImageNet detection validation set despite only having detection data for 44 of the 200 classes. On the 156 classes not in COCO, YOLO9000 gets 16.0 mAP. But YOLO can detect more than just 200 classes; it predicts detections for more than 9000 different object categories. And it still runs in real-time.
Darknet: Open source neural networks in c.
  • Redmon
Redmon, J., "Darknet: Open source neural networks in c." http://pjreddie.com/darknet/ (2013-2016). (Accessed: 12 March 2018).