Figure - available from: Sensors
This content is subject to copyright.
The augmented training images; 0, 1 and 2 represent the unripe, partially ripe and ripe strawberries, respectively.
Source publication
Currently, strawberry harvesting relies heavily on human labour and subjective assessments of ripeness, resulting in inconsistent post-harvest quality. Therefore, the aim of this work is to automate this process and provide a more accurate and efficient way of assessing ripeness. We explored a unique combination of YOLOv7 object detection and augme...
Similar publications
In the era of the 21st century, Deep CNN has proven its potential in crop and fruit disease classification and detection. Diseases have a ruinous effect on the quality and gross production of yields, which is related to the world economy. Proper identification of diseases at early stages may save yields from damage. CNN-based disease identification...
Strawberry quality grading helps producers to better manage inventory, transportation and sales, and improve product market competitiveness. Currently, this work is mostly carried out by manual grading, which is not only inefficient, but also relies on the experience of the grader, which is easy to cause grading errors. Aiming at the complex backgr...
Strawberry is a healthy, beneficial fruit and one of the most valuable exports for most countries. However, diseases could produce poor-quality strawberries and affect the consumer’s health. Thus, quality inspection is a crucial stage in processing production. Convolutional Neural Network (CNN) models can be used to identify specific diseases. Even...
Strawberries, known for their economic significance and rich nutritional value, are cultivated extensively worldwide. However, a host of workers need to be employed every year to identify and categorize the developmental stages of the strawberries in the greenhouses, which is not only time-consuming, inefficient, increasing the cultivation cost, bu...
Introduction
In the context of intelligent strawberry cultivation, achieving multi-stage detection and yield estimation for strawberry fruits throughout their full growth cycle is essential for advancing intelligent management of greenhouse strawberries. Addressing the high rates of missed and false detections in existing object detection algorithm...
Citations
... Strawberry fruits with uneven ripening are often identified as damaged or mistakenly classified as ripe fruits during harvesting or sorting, especially when using computer technology with image processing (Chai et al., 2023;Mohamed, 2021;Ouyang et al., 2013). If such fruits are picked before reaching full color but are allowed to develop their color post-harvest, their market value decreases significantly in storage (Hong and Eum, 2020). ...
We studied the physical and chemical profiles of strawberry fruits (Fragaria × ananassa Duch.) of the varieties “Clery” and “Asia,” which exhibited uneven ripening disorder. In the first part of the study, evenly (uniformly) ripe fruits harvested in different harvest seasons – spring and autumn – were compared. The chemical parameters, including total soluble solids (TSS), individual and total sugars, individual and total organic acids, vitamin C, and phenolic content, varied between the two seasons. However, the values of physical parameters, such as fruit color and firmness, were similar between the seasons. The second objective was to compare the characteristics of differently pigmented parts of unevenly ripe fruits. The higher levels of TSS, individual and total sugars, vitamin C, and phenolic compounds, along with lower levels of organic acids and firmness, were found in the most pigmented part of the fruit. Uneven ripening appeared as the “white shoulder” or the “green tip” in the strawberry fruit. In the variety “Clery,” the “white shoulder” was observed in both harvest seasons, while in the variety “Asia,” the “white shoulder” was observed in spring, and the “green tip” in autumn. In conclusion, uneven ripening in strawberries results in significant chemical differences between fruit parts and harvest seasons. Moreover, different parts of unevenly ripe fruits show different maturity based on physical and chemical parameters.
... You Only Look Once (YOLO) series algorithms, as one of the most popular target detection algorithms at present, have achieved remarkable results in many fields due to their high efficiency and accuracy [16][17][18]. Jackey et al. [19] explored the combination of YOLOv7 with augmented reality technology to detect and visualize the maturity of strawberries. The results show that the mAP value and F1 value of the YOLOv7 model, using transfer learning, fine-tuning, and multi-scale training, are 0.89 and 0.92, respectively. ...
In order to accurately detect the maturity of chili peppers under different lighting and natural environmental scenarios, in this study, we propose a lightweight maturity detection model, YOLOv8-CBSE, based on YOLOv8n. By replacing the C2f module in the original model with the designed C2CF module, the model integrates the advantages of convolutional neural networks and Transformer architecture, improving the model’s ability to extract local features and global information. Additionally, SRFD and DRFD modules are introduced to replace the original convolutional layers, effectively capturing features at different scales and enhancing the diversity and adaptability of the model through the feature fusion mechanism. To further improve detection accuracy, the EIoU loss function is used instead of the CIoU loss function to provide more comprehensive loss information. The results showed that the average precision (AP) of YOLOv8-CBSE for mature and immature chili peppers was 90.75% and 85.41%, respectively, with F1 scores and a mean average precision (mAP) of 81.69% and 88.08%, respectively. Compared with the original YOLOv8n, the F1 score and mAP of the improved model increased by 0.46% and 1.16%, respectively. The detection effect for chili pepper maturity under different scenarios was improved, which proves the robustness and adaptability of YOLOv8-CBSE. YOLOv8-CBSE also maintains a lightweight design with a model size of only 5.82 MB, enhancing its suitability for real-time applications on resource-constrained devices. This study provides an efficient and accurate method for detecting chili peppers in natural environments, which is of great significance for promoting intelligent and precise agricultural management.
... The proposed algorithm 2 presents a practical application of an already trained YOLOv7 object detection model with the Unity AR Foundation framework [33] to allow real-time object detection and interactive visualization of objects in an AR application for educational purposes. The workflow begins with the migration of the YOLOv7 model, which is designed in PyTorch, to the ONNX (Open Neural Network Exchange) format and finally to Unity using it. ...
We propose a novel augmented reality (AR) framework that transforms education with immersive and interactive experiences that enhance student engagement and comprehension. However, existing AR-based learning solutions are built upon generic object detectors that struggle with recognizing domain-specific educational materials, thus limiting their effectiveness. To address this challenge, we introduce a novel fully automated image dataset generation pipeline that synthesizes high-fidelity images from 3D models by varying lighting, camera angles, and background occlusion. This variation enhances the diversity of the dataset, enabling robust training of domain-specific object detectors. The proposed pipeline stands out from existing methods because it provides scalability and adaptability features that allow researchers to build customized educational datasets. The study involved generating a dataset and evaluating it using four state-of-the-art object detection models: Faster R-CNN, SSD, YOLOv5n, and YOLOv7. The YOLOv7 detection model reached an accuracy of 97.2% with 99.5% mAP@0.5 and performed at a real-time speed of 45 frames per second (FPS) making it the best choice for AR applications. To assess the educational impact of our system, we conducted a pilot study involving 210 elementary students. The results showed notable improvements in learning outcomes: fourth graders’ scores increased from 68.78 ± 10.85 to 90.96 ± 11.70, while fifth graders improved from 62.43 ± 11.53 to 75.43 ± 12.53. Comprehensive statistical analyses, including ANOVA, regression, and paired t-tests, confirmed that our approach significantly enhances both academic performance and student engagement when compared to traditional learning methods. These findings demonstrate that our domain-focused data pipeline and optimized object detection framework effectively bridge the gap between deep learning research and AR real-world classroom implementation, offering a highly scalable and transformative solution for AR-based education.
... Furthermore, XR also plays a role in decisionmaking support, as seen in [50]'s work on creating climatesmart and profitable farms using VR. The development of more advanced systems, such as the real-time detection of strawberry ripeness using AR [45], reflects a growing trend toward combining XR with cuttingedge technologies like AI and the Internet of Things (IoT). In addition, cross-disciplinary applications are also on the rise, as illustrated by [37] use of AR and VR for immersive agricultural education through gamification. ...
The agricultural sector confronts critical challenges such as climate change, overpopulation, and resource shortages, necessitating urgent innovative solutions. In this context, Extended Reality (XR) technologies have emerged as a transformative technology, offering groundbreaking approaches to modern farming practices. This study conducts a thorough literature review to explore the integration of XR technologies in agriculture, livestock farming, and aquaculture. By categorizing studies based on focus areas, application types, and human-machine interaction dimensions, it identifies emerging trends, highlights critical gaps, and proposes future direction for advancing these fields. The analysis revealed a significant increase in XR technology adoption over the past two years, highlighting its potential within the sector. The findings suggest that XR technologies can enhance productivity, improve training methods, and facilitate remote collaboration in agricultural settings. However, future research should focus on addressing health, privacy, and security concerns to enhance seamless and immersive XR experiences, thereby maximizing the potential benefits of XR technology in the primary sector. Therefore, these technologies will transform the agriculture industry, fostering a resilient and sustainable food system to meet the demands of a growing global population.
... For instance, financial analysts can use AR to visualize various financial scenarios and outcomes in three dimensions, improving their ability to make informed decisions based on real-time data. The synergy between ML and AR has proven to be particularly powerful in financial analytics [6], [7]. By combining ML's predictive capabilities with AR's immersive visualization tools, financial professionals can achieve a more comprehensive and effective approach to data analysis. ...
The integration of Machine Learning (ML) algorithms with real-time enterprise finance systems and Augmented Reality (AR) applications represents a significant leap forward in financial analytics and data visualization. Efficient ML algorithms are essential for processing and analyzing large volumes of financial data in real time, enabling enterprises to make informed decisions based on up-to-date information. These algorithms enhance predictive accuracy, streamline risk management, and improve fraud detection by learning from historical data and adapting to emerging trends. In parallel, AR technology transforms the way financial data is visualized and interacted with. By overlaying digital information onto the physical environment, AR provides immersive and interactive experiences that make complex financial metrics more accessible and understandable. This combination of ML and AR in real-time environments allows for dynamic and intuitive data visualization, enabling financial professionals to explore and manipulate data in innovative ways. Efficient ML algorithms, such as those based on deep learning and reinforcement learning, are optimized for rapid data processing and analysis. These algorithms support real-time decision-making by delivering precise insights and predictions on financial trends, market behaviors, and risk factors. The integration of AR further enhances these capabilities by presenting data in a spatial and interactive format, allowing users to visualize trends, scenarios, and outcomes in a more engaging and practical manner. The synergy between ML and AR in real-time finance applications not only improves the accuracy and efficiency of financial analysis but also facilitates more strategic and personalized financial management. As these technologies continue to evolve, their combined application is expected to drive further innovation, setting new standards for financial analytics and decision-making. This approach delivers accurate, timely insights and interactive experiences, paving the way for more informed and effective financial management.
... The model used a single-layer feature map from the ResNet101 backbone network and an encoder-decoder structure to detect and classify the green fruit. In the study of [22], the article discusses the development and implementation of a real-time strawberry ripeness detection system using augmented reality (AR) and deep learning. The researchers used the YOLOv7 deep learning model for object detection and classification of strawberry ripeness levels (unripe, partially ripe, ripe) exploring the use of multi-scale training and a lightweight YOLOv7-tiny model to balance detection accuracy and speed. ...
Accurate identification of strawberries during their maturing stages is crucial for optimizing yield management, and pest control, and making informed decisions related to harvest and post-harvest logistics. This study evaluates the performance of YOLOv8 model configurations for instance segmentation of strawberries into ripe and unripe stages in an open field environment. The YOLOv8n model demonstrated superior segmentation accuracy with a mean Average Precision (mAP) of 80.9\%, outperforming other YOLOv8 configurations. In terms of inference speed, YOLOv8n processed images at 12.9 milliseconds, while YOLOv8s, the least-performing model, processed at 22.2 milliseconds. Over 86 test images with 348 ground truth labels, YOLOv8n detected 235 ripe fruit classes and 51 unripe fruit classes out of 251 ground truth ripe fruits and 97 unripe ground truth labels, respectively. In comparison, YOLOv8s detected 204 ripe fruits and 37 unripe fruits. Overall, YOLOv8n achieved the fastest inference speed of 24.2 milliseconds, outperforming YOLOv8s, YOLOv8m, YOLOv8l, and YOLOv8x, which processed images at 33.0 milliseconds, 44.3 milliseconds, 53.6 milliseconds, and 62.5 milliseconds, respectively. These results underscore the potential of advanced object segmentation algorithms to address complex visual recognition tasks in open-field agriculture effectively to address complex visual recognition tasks in open-field agriculture effectively.
... The aforementioned YOLO-based models have effectively improved the detection speed, but there is still room for improvement in the detection accuracy. Chai et al. [21] proposed a novel strawberry detection algorithm based on a unique combination of YOLOv7 and augmented reality technology, which achieved a high F1 score of 92%. However, the method's accuracy in natural environments cannot be guaranteed, as the detection environment was limited to a greenhouse. ...
The recognition and localization of strawberries are crucial for automated harvesting and yield prediction. This article proposes a novel RTF-YOLO (RepVgg-Triplet-FocalLoss-YOLO) network model for real-time strawberry detection. First, an efficient convolution module based on structural reparameterization is proposed. This module was integrated into the backbone and neck networks to improve the detection speed. Then, the triplet attention mechanism was embedded into the last two detection heads to enhance the network’s feature extraction for strawberries and improve the detection accuracy. Lastly, the focal loss function was utilized to enhance the model’s recognition capability for challenging strawberry targets, which thereby improves the model’s recall rate. The experimental results demonstrated that the RTF-YOLO model achieved a detection speed of 145 FPS (frames per second), a precision of 91.92%, a recall rate of 81.43%, and an mAP (mean average precision) of 90.24% on the test dataset. Relative to the baseline of YOLOv5s, it showed improvements of 19%, 2.3%, 4.2%, and 3.6%, respectively. The RTF-YOLO model performed better than other mainstream models and addressed the problems of false positives and false negatives in strawberry detection caused by variations in illumination and occlusion. Furthermore, it significantly enhanced the speed of detection. The proposed model can offer technical assistance for strawberry yield estimation and automated harvesting.
... With the development of AI, sensor technology, and computer vision technology, research has been conducted to apply these technologies to the agricultural field, such as smart farms, to create high added-value [28][29][30][31][32][33][34][35]. In particular, for the system that determines the ripeness of fruits, many studies have been carried out utilizing AI and computer vision technology, as it is often necessary to judge the ripeness by external factors such as color, shape, etc. [28][29][30]. ...
... Strawberries have been widely employed in many studies as a fruit for ripeness detection research based on computer vision technology due to their distinct color changes depending on the degree of ripeness [31][32][33][34][35]. The authors in [31] utilized the YOLOv7 model for object detection and ripeness measurement of strawberries. ...
... Strawberries have been widely employed in many studies as a fruit for ripeness detection research based on computer vision technology due to their distinct color changes depending on the degree of ripeness [31][32][33][34][35]. The authors in [31] utilized the YOLOv7 model for object detection and ripeness measurement of strawberries. They also visualized the location and ripeness with augmented reality (AR) technology. ...
Image analysis-based artificial intelligence (AI) models leveraging convolutional neural networks (CNN) take a significant role in evaluating the ripeness of strawberry, contributing to the maximization of productivity. However, the convolution, which constitutes the majority of the CNN models, imposes significant computational burdens. Additionally, the dense operations in the fully connected (FC) layer necessitate a vast number of parameters and entail extensive external memory access. Therefore, reducing the computational burden of convolution operations and alleviating memory overhead is essential in embedded environment. In this paper, we propose a strawberry ripeness classification system utilizing a convolution-based feature extractor (CoFEx) for accelerating convolution operations and an edge AI processor, Intellino, for replacing FC layer operations. We accelerated feature map extraction utilizing the CoFEx constructed with systolic array (SA) and alleviated the computational burden and memory overhead associated with the FC layer operations by replacing them with the k-nearest neighbors (k-NN) algorithm. The CoFEx and the Intellino both were designed with Verilog HDL and implemented on a field-programmable gate array (FPGA). The proposed system achieved a high precision of 93.4%, recall of 93.3%, and F1 score of 0.933. Therefore, we demonstrated a feasibility of the strawberry ripeness classification system operating in an embedded environment.
Recent advances in artificial intelligence and computer vision have led to significant progress in the use of agricultural technologies for yield prediction, pest detection, and real-time monitoring of plant conditions. However, collecting large-scale, high-quality image datasets in the agriculture sector remains challenging, particularly for specialized datasets such as plant disease images. This study analyzed the effects of the image size (320–640+) and the number of labels on the performance of a YOLO-based object detection model using diverse agricultural datasets for strawberries, tomatoes, chilies, and peppers. Model performance was evaluated using the intersection over union and average precision (AP), where the AP curve was smoothed using the Savitzky–Golay filter and EEM. The results revealed that increasing the number of labels improved the model performance to a certain degree, after which the performance gradually diminished. Furthermore, while increasing the image size from 320 to 640 substantially enhanced the model performance, additional increases beyond 640 yielded only marginal improvements. However, the training time and graphics processing unit usage scaled linearly with increasing image sizes, as larger size images require greater computational resources. These findings underscore the importance of an optimal strategy for selecting the image size and label quantity under resource constraints in real-world model development.
