ArticlePublisher preview available

MiniTomatoNet: a lightweight CNN for tomato leaf disease recognition on heterogeneous FPGA-SoC

Authors:
Theodora Sanida at University of Western Macedonia
Theodora Sanida
Minas Dasygenis at University of Western Macedonia
Minas Dasygenis
Read publisher preview
Download citation
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

Recognition of leaf diseases in agriculture is considered a significant aspect of ensuring food quantity, quality, and production. In general, crop leaves are susceptible and fragile to various diseases such as leaf mold, target spot, late blight, bacterial spot or early blight of tomato plants. However, these tomato plant diseases are challenging to recognize, and early diagnosis is vital. At the same time, the continuous growth of convolutional neural network (CNN) approaches has significantly assisted plant disease diagnosis, providing a robust mechanism with highly accurate results. On the other hand, the number of unhealthy leaf images collected is often unbalanced, and diagnosing diseases with such an unbalanced data set is complicated. So, numerous models for tomato disease diagnosis based on CNN models have been proposed. However, none overcomes the class imbalance problem and, as a result, does not generate findings with impartial accuracy. This article presents an efficient and robust solution for the heterogeneous PYNQ-Z1 board. Optimization techniques-including loop unrolling, pipelining, array partitioning, and loop flattening-enhance the computation speed across the network’s convolutional, fully connected, and max-pooling layers. The presented CNN approach comprises an 8-layer network termed MiniTomatoNet. This network is characterized by its streamlined structure, possessing only under 23 K parameters with all weights and biases and occupying a memory of 89.51 KB. In addition, the model trains with a re-weighted focal loss function and achieves 97.63% accuracy and 98.51% AUC score; the inference rate speed is 0.068 s per frame, and the power consumption is 2.35 W. Finally, the model is efficient, low power, robust, high accuracy and fast speed, making it a promising solution for diagnosing tomato diseases.
Figures - available from: The Journal of Supercomputing
This content is subject to copyright. Terms and conditions apply.
A preview of this full-text is provided by Springer Nature.
Learn more
Content available from The Journal of Supercomputing
This content is subject to copyright. Terms and conditions apply.
Vol.:(0123456789)
The Journal of Supercomputing (2024) 80:21837–21866
https://doi.org/10.1007/s11227-024-06301-8
1 3
MiniTomatoNet: alightweight CNN fortomato leaf disease
recognition onheterogeneous FPGA‑SoC
TheodoraSanida1· MinasDasygenis1
Accepted: 9 June 2024 / Published online: 17 June 2024
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature
2024
Abstract
Recognition of leaf diseases in agriculture is considered a significant aspect of
ensuring food quantity, quality, and production. In general, crop leaves are suscepti-
ble and fragile to various diseases such as leaf mold, target spot, late blight, bacterial
spot or early blight of tomato plants. However, these tomato plant diseases are chal-
lenging to recognize, and early diagnosis is vital. At the same time, the continuous
growth of convolutional neural network (CNN) approaches has significantly assisted
plant disease diagnosis, providing a robust mechanism with highly accurate results.
On the other hand, the number of unhealthy leaf images collected is often unbal-
anced, and diagnosing diseases with such an unbalanced data set is complicated.
So, numerous models for tomato disease diagnosis based on CNN models have been
proposed. However, none overcomes the class imbalance problem and, as a result,
does not generate findings with impartial accuracy. This article presents an efficient
and robust solution for the heterogeneous PYNQ-Z1 board. Optimization tech-
niques-including loop unrolling, pipelining, array partitioning, and loop flattening-
enhance the computation speed across the network’s convolutional, fully connected,
and max-pooling layers. The presented CNN approach comprises an 8-layer network
termed MiniTomatoNet. This network is characterized by its streamlined structure,
possessing only under 23 K parameters with all weights and biases and occupying
a memory of 89.51 KB. In addition, the model trains with a re-weighted focal loss
function and achieves 97.63% accuracy and 98.51% AUC score; the inference rate
speed is 0.068s per frame, and the power consumption is 2.35 W. Finally, the model
is efficient, low power, robust, high accuracy and fast speed, making it a promising
solution for diagnosing tomato diseases.
Keywords Heterogeneous FPGA-SoC· Lightweight CNN· Class imbalanced
learning· Tomato disease recognition
Extended author information available on the last page of the article
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
... Huang et al. (2019) proposed an end-to-end plant disease diagnosis model based on deep neural networks that could reliably identify plant types and diseases. Sanida and Dasygenis (2024) proposed a lightweight convolutional neural network for tomato leaf disease identification, achieving an accuracy of 97.63% and an AUC score of 98.51%. Ni et al. (2023) introduced a tomato leaf disease recognition model based on ResNet18, enhanced by the addition of a squeeze-and-excitation module, which attained an average recognition accuracy of 99.63% on the publicly available PlantVillage dataset. ...
Ultra-lightweight tomatoes disease recognition method based on efficient attention mechanism in complex environment
Article
Full-text available
  • Feb 2025
A variety of diseased leaves and background noise types are present in images of diseased tomatoes captured in real-world environments. However, existing tomato leaf disease recognition models are limited to recognizing only a single leaf, rendering them unsuitable for practical applications in real-world scenarios. Additionally, these models consume significant hardware resources, making their implementation challenging for agricultural production and promotion. To address these issues, this study proposes a framework that integrates tomato leaf detection with leaf disease recognition. This framework includes a leaf detection model designed for diverse and complex environments, along with an ultra-lightweight model for recognizing tomato leaf diseases. To minimize hardware resource consumption, we developed five inverted residual modules coupled with an efficient attention mechanism, resulting in an ultra-lightweight recognition model that effectively balances model complexity and accuracy. The proposed network was trained on a dataset collected from real environments, and 14 contrasting experiments were conducted under varying noise conditions. The results indicate that the accuracy of the ultra-lightweight tomato disease recognition model, which utilizes the efficient attention mechanism, is 97.84%, with only 0.418 million parameters. Compared to traditional image recognition models, the model presented in this study not only achieves enhanced recognition accuracy across 14 noisy environments but also significantly reduces the number of required model parameters, thereby overcoming the limitation of existing models that can only recognize single disease images.
View
View
ECVNet: A Fusion Network of Efficient Convolutional Neural Networks and Visual Transformers for Tomato Leaf Disease Identification
Article
Full-text available
  • Dec 2024
Tomato leaf diseases pose a significant threat to plant growth and productivity, necessitating the accurate identification and timely management of these issues. Existing models for tomato leaf disease recognition can primarily be categorized into Convolutional Neural Networks (CNNs) and Visual Transformers (VTs). While CNNs excel in local feature extraction, they struggle with global feature recognition; conversely, VTs are advantageous for global feature extraction but are less effective at capturing local features. This discrepancy hampers the performance improvement of both model types in the task of tomato leaf disease identification. Currently, effective fusion models that combine CNNs and VTs are still relatively scarce. We developed an efficient CNNs and VTs fusion network named ECVNet for tomato leaf disease recognition. Specifically, we first designed a Channel Attention Residual module (CAR module) to focus on channel features and enhance the model’s sensitivity to the importance of feature channels. Next, we created a Convolutional Attention Fusion module (CAF module) to effectively extract and integrate both local and global features, thereby improving the model’s spatial feature extraction capabilities. We conducted extensive experiments using the Plant Village dataset and the AI Challenger 2018 dataset, with ECVNet achieving state-of-the-art recognition performance in both cases. Under the condition of 100 epochs, ECVNet achieved an accuracy of 98.88% on the Plant Village dataset and 86.04% on the AI Challenger 2018 dataset. The introduction of ECVNet provides an effective solution for the identification of plant leaf diseases.
View
High-speed YOLOv4-tiny hardware accelerator for self-driving automotive
Article
Full-text available
  • Oct 2023
  • J SUPERCOMPUT
Object detection is an important area in self-driving automotive. The YOLO algorithm and its well-embedded implementation is a promising solution for object detection. In this paper, a novel hardware implementation of YOLOv4-tiny object detection has been presented on FPGA. Since the YOLO network has many calculations and parameters, an 8-bit and 5-bit fixed-point format for data and weight has been proposed to reduce resources and memory. To compensate for the accuracy, the best decimal point position in different layers is extracted using a Genetic Algorithm for network quantization. Also, a technique for performing two multiplications simultaneously with completely different operands with one DSP BLOCK has been presented, which has increased the network execution speed by 1.8 times. We implemented our design on the Xilinx Zynq ZC706 FPGA. Our accelerator can execute YOLOv4-tiny at the resolution of 416 × 416 at the speed of 55 FPS and achieve an accuracy of 79%. Compared to the state of the art, the FPS has increased by 13%, while the accuracy has decreased by only 3%, and also the proposed scheme uses fewer DSPs, which shows the resource utilization of the proposed architecture is better than previous works.
View
Deep multi-scale convolutional neural networks for automated classification of multi-class leaf diseases in tomatoes
Article
Full-text available
  • Oct 2023
  • NEURAL COMPUT APPL
Deep learning techniques have gained immense popularity recently because of their remarkable capacity to learn complex patterns and features from large datasets. These techniques have revolutionized many fields by achieving advanced performance in various tasks. The availability of large datasets and the advancement of computing resources have enabled deep learning models to perform well in solving challenging problems. As a result, they have become an essential tool in many industries, including agriculture. The application of deep learning in agriculture has great potential for increasing productivity, reducing costs, and improving sustainability by aiding in the early identification and prevention of plant leaf diseases, optimizing crop yields, and facilitating precision agriculture. This paper suggests using a novel approach to automatically classify multi-class leaf diseases in tomatoes using a deep multi-scale convolutional neural network (DMCNN). The proposed DMCNN architecture consists of parallel streams of convolutional neural networks at different scales, which get merged at the end to form a single output. The images of tomato leaves are preprocessed using data augmentation techniques and fed into the DMCNN model to classify disease. The proposed approach is evaluated on a dataset of tomato plant images containing 10 distinct classes of diseases and compared with different existing models. The research results reveal that the suggested DMCNN model performs better than other models in terms of accuracy, precision, recall, and F1 score. Furthermore, the proposed model reported an overall accuracy of 99.1%, which is higher than the accuracy of existing models tested on the same dataset. The study demonstrates the potential of deep learning techniques for automated disease classification in agriculture, which can aid in early disease detection and prevent crop loss.
View
A real-time application-based convolutional neural network approach for tomato leaf disease classification
Article
Full-text available
  • Aug 2023
Early diagnosis and treatment of tomato leaf diseases increase a plant's production volume, efficiency, and quality. Misdiagnosis of disease by farmers can lead to an inadequate treatment strategy that hurts the tomato plants and agroecosystem. Therefore, it is crucial to detect the disease precisely. Finding a rapid, accurate approach to take care of the issue of misdiagnosis and early disease identification will be advantageous to the farmers. This study proposed a lightweight custom convolutional neural network (CNN) model and utilized transfer learning (TL)-based models VGG-16 and VGG-19 to classify tomato leaf diseases. In this study, eleven classes, one of which is healthy, are used to simulate various tomato leaf diseases. In addition, an ablation study has been performed in order to find the optimal parameters for the proposed model. Furthermore, evaluation metrics have been used to analyze and compare the performance of the proposed model with the TL-based model. The proposed model, by applying data augmentation techniques, has achieved the highest accuracy and recall of 95.00% among all the models. Finally, the best-performing model has been utilized in order to construct a Web-based and Android-based end-to-end (E2E) system for tomato cultivators to classify tomato leaf disease.
View
Machine learning in agriculture: a review of crop management applications
Article
Full-text available
Machine learning has created new opportunities for data-intensive study in interdisciplinary domains as a result of the advancement of big data technologies and high-performance computers. Search engines, email spam filters, websites that offer personalized recommendations, banking software that alerts users to suspicious activity, and a plethora of smartphone apps that perform tasks like voice recognition, image recognition, and natural language processing are just a few examples of the online and offline services that have incorporated machine learning in recent years. One of the most crucial areas where machine learning applications still has to be investigated is agriculture, which directly affects people’s well-being. In this article, a literature review on machine learning algorithms used in agriculture is presented. The proposed paper deal with various crop management applications which are categorised into five parts i.e., Weed and pest detection, Plant disease detection, Stress detection in plants, Smart farms or automation in farms and the last one is Crop yield estimation and prediction. The articles’ filtering and categorization show how machine learning may improve agriculture. This article examines machine learning breakthroughs in agriculture. This paper’s findings show that by using novel machine learning approaches, models may achieve improved accuracy and shorter inference time for real-world applications.
View
Hardware acceleration of YOLOv7-tiny using high-level synthesis tools
Article
Full-text available
  • Jun 2023
FPGAs have emerged as a promising platform for implementing neural networks due to their reconfigurability, parallelism, and low power consumption. Nonetheless, designing and optimizing FPGA-based neural network accelerators is a complex and time-consuming task with register transfer level (RTL) languages. High-level synthesis (HLS) tools provide a higher level of abstraction for FPGA design, enabling designers to concentrate on top-level design aspects, such as algorithms, rather than low-level hardware implementation details. One of the state-of-the-art object detection networks is you look only once (YOLO) network series which is constructed using different neural network technologies using cross-stage connections and feature extraction techniques like pyramid networks. In this paper, we propose a method for the implementation of YOLOv7-tiny network on FPGAs using HLS tools. We present a comprehensive analysis of the performance and resource utilization of FPGA-based neural network accelerators. Our methods show excellent results for real-time application requirements such as latency. Specifically, our work reduces the usage of digital signal processing (DSP) units by 90% and it saves up to 60% of flip-flops compared to state-of-the-art designs, while achieving competitive usage of block RAM and look-up tables. Additionally, the achieved design latency of 15 ms is extremely suitable for real-time applications. Also we will propose a method for BRAM utilization method and off-chip memory access.
View
Artificial intelligence-based solutions for climate change: a review
Article
Full-text available
  • Jun 2023
Climate change is a major threat already causing system damage to urban and natural systems, and inducing global economic losses of over $500 billion. These issues may be partly solved by artificial intelligence because artificial intelligence integrates internet resources to make prompt suggestions based on accurate climate change predictions. Here we review recent research and applications of artificial intelligence in mitigating the adverse effects of climate change, with a focus on energy efficiency, carbon sequestration and storage, weather and renewable energy forecasting, grid management, building design, transportation, precision agriculture, industrial processes, reducing deforestation, and resilient cities. We found that enhancing energy efficiency can significantly contribute to reducing the impact of climate change. Smart manufacturing can reduce energy consumption, waste, and carbon emissions by 30–50% and, in particular, can reduce energy consumption in buildings by 30–50%. About 70% of the global natural gas industry utilizes artificial intelligence technologies to enhance the accuracy and reliability of weather forecasts. Combining smart grids with artificial intelligence can optimize the efficiency of power systems, thereby reducing electricity bills by 10–20%. Intelligent transportation systems can reduce carbon dioxide emissions by approximately 60%. Moreover, the management of natural resources and the design of resilient cities through the application of artificial intelligence can further promote sustainability.
View
A Survey on Neural Network Hardware Accelerators
Article
  • Aug 2024
Artificial intelligence hardware accelerator is an emerging research for several applications and domains. The hardware accelerator’s direction is to provide high computational speed with retaining low-cost and high learning performance. The main challenge is to design complex machine learning models on hardware with high performance. This paper presents a thorough investigation into machine learning accelerators and associated challenges. It describes a hardware implementation of different structures such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Artificial Neural Network (ANN). The challenges such as speed, area, resource consumption, and throughput are discussed. It also presents a comparison between the existing hardware design. Lastly, the paper describes the evaluation parameters for a machine learning accelerator in terms of learning & testing performance and hardware design.
View
FPGA-based Deep Learning Inference Accelerators: Where Are We Standing?
Article
  • Sep 2023
Recently, artificial intelligence applications have become part of almost all emerging technologies around us. Neural networks, in particular, have shown significant advantages and have been widely adopted over other approaches in machine learning. In this context, high processing power is deemed a fundamental challenge and a persistent requirement. Recent solutions facing such a challenge deploy hardware platforms to provide high computing performance for neural networks and deep learning algorithms. This direction is also rapidly taking over the market. Here, FPGAs occupy the middle ground regarding flexibility, reconfigurability, and efficiency compared to general-purpose CPUs, GPUs, on one side, and manufactured ASICs on the other. FPGA-based accelerators exploit the features of FPGAs to increase the computing performance for specific algorithms and algorithm features. Filling a gap, we provide holistic benchmarking criteria and optimization techniques that work across several classes of deep learning implementations. This paper summarizes the current state of deep learning hardware acceleration: More than 120 FPGA-based neural network accelerator designs are presented and evaluated based on a matrix of performance and acceleration criteria, and corresponding optimization techniques are presented and discussed. In addition, the evaluation criteria and optimization techniques are demonstrated by benchmarking ResNet-2 and LSTM-based accelerators.
View
Tomato Plant Leaf Disease Segmentation and Multiclass Disease Detection Using Hybrid Optimization Enabled Deep Learning
Article
  • Aug 2023
Production of crops is increasing day by day in agriculture sectors. The insecurity of food is a main reason of plant disease and is a main global issue that humans face these days. With the design of contemporary environmental agriculture, more focus is devised for yielding the crop and elevating its quality. The occurrence of crops has elevated in years and the kind of disease has become more and more complex. The disease in plants and the pernicious insects are the major risks in agriculture field. Thus, earlier discovery and treatment of this disease are imperative. The major design of Deep Learning (DL) model helped in detecting the plant disease and grants a dynamic tool with accurate results. This paper presents DL-assisted technique for detecting and classifying the tomato disease and used deep batch-normalized eLu Alex Net (DbneAlexnet) for classifying the tomato plant leaves. Initially, tomato plant leaf images are taken as an input from specific dataset represented and it is subjected to preprocessing phase to eliminate unwanted distortions using anisotropic filtering. Then, the segmentation is carried out using U-net, which is trained by Gradient-Golden search optimization (Gradient-GSO) Algorithm and it is incorporation of both Golden search optimization (GSO) and Gradient concept. Thereafter the segmented image is given to image augmentation process, where position augmentation and color augmentation are considered. Finally, the multiclass plant leaf disease is classified using DbneAlexnet and is trained using proposed Gradient Jaya- Golden search optimization (GJ-GSO). Here, the GJ-GSO is devised with the integration of Gradient concept, Jaya algorithm, and GSO algorithm. The proposed GJ-GSO-based DbneAlexnet outperformed highest accuracy of 92.4%, True positive rate (TPR) of 91.9%, True negative rate (TNR) of 92.2% and smallest False Positive Rate (FPR) of 0.078. Hence, the technique with unified segmentation and classification is effectual for identifying the plant disease and the empirical research verifies the benefits of the developed model.
View
View