Fig 2 - uploaded by Jinqiang Bai
Content may be subject to copyright.
Source publication
This paper presents a novel garbage pickup robot which operates on the grass. The robot is able to detect the garbage accurately and autonomously by using a deep neural network for garbage recognition. In addition, with the ground segmentation using a deep neural network, a novel navigation strategy is proposed to guide the robot to move around. Wi...
Contexts in source publication
Context 1
... shown in Fig. 2, the complete system of this robot includes five major parts: (1) sensors for environment perception, (2) robot base, (3) controller, (4) manipulator, and (5) garbage ...
Context 2
... test the cleaning efficiency, we have conducted two experiments on a playground (85.4 m * 73 m). Experiment 1 is that the robot cleans the whole area according to the planning path, as shown in Fig. 12; Experiment 2 is that the robot randomly travels to clean the whole area, as shown in Fig. 13. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCE.2018.2859629, IEEE Transactions on Consumer ...
Similar publications
Image segmentation using active contour models to improve image processing enhances object detection. Various segmentation methods have been proposed over in the past to improve the accuracy of segmentation results, such as clustering, edge-based, region-based, template matching, and hybrid methods. However, the image segmentation results of these...
Citations
... Mobile robots are emerging in different sectors, such as companies, industries, hospitals, institutions, agriculture, and homes, to improve services and daily activities [1][2][3]. As technology advances, the demand for mobile robots has increased due to their tasks and services, such as carrying heavy objects, cleaning, and search and rescue missions [4,5]. For example, mobile robots, which are adept at smart path planning and collision risk reduction, are ideal for intelligent logistics and distribution applications [6][7][8]. ...
Mobile robots have been widely engaged in many fields. To obtain the precise and consistent localization of mobile robots, the Global Navigation Satellite System (GNSS) is often employed. With the continuous development and modernization of GNSS, more tracked satellites can be used for multi-GNSS positioning calculation, which can improve the positioning performance and enhance accuracy. However, it also increases computational complexity. Therefore, a satellite selection method, which selects a subset from all visible satellites, is necessary. In multi-GNSS positioning, the geometric dilution of precision (GDOP) is an essential metric for satellite selection. However, the traditional traversal method requires a large amount of solution resources. In this paper, we proposed an improved genetic algorithm for satellite selection. By defining the maturity factor (MF) to guide the crossover and mutation operators, the search performance is guaranteed while reducing unnecessary crossover and mutation operations, thus reducing the search time. By adopting the previous epoch optimal individual inheritance strategy, the satellite selection results of subsequent epochs under continuous epochs have been improved. The experimental results verify the effectiveness of the proposed method.
... In 1999, a project was started at the Lulea University of Technology, the aim of which was to develop a metal scrap recycling system using mechanical forming properties [12]. Using a Bayesian computational framework, the system incorporated strain and outline features into the Flickr material database [13] Jinqiang Bai et al. a garbage collection robot with an innovative design that can independently detect garbage through deep neural networks [14]. Artzai Picon et al. presented a novel fuzzy spectral and spatial classification algorithm that effectively reduced the dimensionality of hyperspectral data by constructing spectral fuzzy sets. ...
In the pursuit of enhancing waste management through technology, this study presents a comprehensive analysis of various Deep Learning models applied to garbage classification. Central to this research is the development and evaluation of an Advanced Custom Convolutional Neural Network (CNN) model, specifically designed for a multi-class garbage classification task. This custom model is meticulously compared against six pre-trained models: ResNet50, VGG16, DenseNet121, MobileNetV2, Ef-ficientNetB0, and AlexNet, each a stalwart in the field of image classification. Employing techniques like Adam optimization and strategic data augmentation, the study aims to optimize performance, especially critical in datasets with diverse and complex waste categories. Remarkably, the Advanced Custom CNN demonstrates exceptional validation accuracies, out-performing the established pre-trained models. This achievement underscores the effectiveness of custom-tailored models in specific applications like waste classification. The findings of this research highlight the potential of customized Deep Learning models in real-world environmental applications. They also shed light on the significance of model architecture choices in achieving high accuracy in specialized tasks. This paper not only contributes a novel approach to waste classification but also serves as a valuable resource for future explorations in applying AI for environmental sustainability.
... The results of the project practice summarize several forward-looking strategic plans for AI-driven service design and management in future urban solid waste management. In addition, this paper discusses the environmental constraints, challenges faced, driving factors, and future research Recognition accuracy is 94%, even without path planning Bai et al. (2018) [21]; Lee, K.-F. (2023) [6] Seven types of garbage [11] Therefore, it is imperative to conduct further research to unravel the multiple roles of AI technology or AI agents in waste classification management systems. ...
... The results of the project practice summarize several forward-looking strategic plans for AI-driven service design and management in future urban solid waste management. In addition, this paper discusses the environmental constraints, challenges faced, driving factors, and future research Recognition accuracy is 94%, even without path planning Bai et al. (2018) [21]; Lee, K.-F. (2023) [6] Seven types of garbage [11] Therefore, it is imperative to conduct further research to unravel the multiple roles of AI technology or AI agents in waste classification management systems. ...
This paper proposes a framework for AI-driven municipal solid waste classification service design and management, with an emphasis on advancing sustainable urban development. This study uses narrative research and case study methods to delve into the benefits of AI technology in waste classification systems. The framework includes intelligent recognition, management strategies, AI-based waste classification technologies, service reforms, and AI-powered customer involvement and education. Our research indicates that AI technology can improve accuracy, efficiency, and cost-effectiveness in waste classification, contributing to environmental sustainability and public health. However, the effectiveness of AI applications in diverse city contexts requires further verification. The framework holds theoretical and practical significance, offering insights for future service designs of waste management and promoting broader goals of sustainable urban development.
... Sensor Algorithm [24] Hindrance evasion LiDAR LOS-based GNC Algorithm [25] Hindrance avoidance Navigation radar IACO Algorithm [26] Hindrance detection Vision Sensors Skip-ENet Algorithm [27] Watermark and hindrance recognition Camera U-net CNN Algorithm [28] Hindrance evasion LiDAR ATESOA algorithm [29] Hindrance evasion Microwave radar, 4 G camera - [29] Water quality monitoring Chl-aturbidity, water dissolved oxygen, water conductivity, oxidation-reduction potential, water temperature, salinity, Ph sensors - [30] Hindrance avoidance Laser scanner. Sonar, Alimeter VFH + algorithm [30] Water eminence Observation Chemical sensors (Hg, Cr, Cd and dispersed oil) - [31] Water eminence Observation Ph sensors, ORP sensor, salinity sensor, dissolved oxygen prob - [32] Water eminence Observation Ph sensor, TDS sensor, turbidity sensor Ensemble learning algorithm [33] Water eminence Observation CTD, DO, Ph, Pco2, nitrate, chlorophyll fluorance, CDOM, turbidity, DOC sensors - [34] Water surface cleaning Vision module, Grasper YOLOv3 algorithm [35] Water surface cleaning Conveyor belt - [36] Water surface cleaning Camera, waste scooper - [37] Water surface cleaning Camera, collection box YOLO v3 algorithm [38] Hindrance avoidance Ultrasonic sensors Threshold-based obstacle avoidance algorithm [38] Water quality monitoring Ph sensor - [38] Water surface cleaning Vision sensor, salvage net Hue-based colour filtering algorithm depth, oxygen dissolved, nitrate, colour dissolved organic matter, and turbidity. In this way, the autonomous cars increase the scope of their time-and space-based surveillance. ...
... Sensor Algorithm [24] Hindrance evasion LiDAR LOS-based GNC Algorithm [25] Hindrance avoidance Navigation radar IACO Algorithm [26] Hindrance detection Vision Sensors Skip-ENet Algorithm [27] Watermark and hindrance recognition Camera U-net CNN Algorithm [28] Hindrance evasion LiDAR ATESOA algorithm [29] Hindrance evasion Microwave radar, 4 G camera - [29] Water quality monitoring Chl-aturbidity, water dissolved oxygen, water conductivity, oxidation-reduction potential, water temperature, salinity, Ph sensors - [30] Hindrance avoidance Laser scanner. Sonar, Alimeter VFH + algorithm [30] Water eminence Observation Chemical sensors (Hg, Cr, Cd and dispersed oil) - [31] Water eminence Observation Ph sensors, ORP sensor, salinity sensor, dissolved oxygen prob - [32] Water eminence Observation Ph sensor, TDS sensor, turbidity sensor Ensemble learning algorithm [33] Water eminence Observation CTD, DO, Ph, Pco2, nitrate, chlorophyll fluorance, CDOM, turbidity, DOC sensors - [34] Water surface cleaning Vision module, Grasper YOLOv3 algorithm [35] Water surface cleaning Conveyor belt - [36] Water surface cleaning Camera, waste scooper - [37] Water surface cleaning Camera, collection box YOLO v3 algorithm [38] Hindrance avoidance Ultrasonic sensors Threshold-based obstacle avoidance algorithm [38] Water quality monitoring Ph sensor - [38] Water surface cleaning Vision sensor, salvage net Hue-based colour filtering algorithm depth, oxygen dissolved, nitrate, colour dissolved organic matter, and turbidity. In this way, the autonomous cars increase the scope of their time-and space-based surveillance. ...
... Sensor Algorithm [24] Hindrance evasion LiDAR LOS-based GNC Algorithm [25] Hindrance avoidance Navigation radar IACO Algorithm [26] Hindrance detection Vision Sensors Skip-ENet Algorithm [27] Watermark and hindrance recognition Camera U-net CNN Algorithm [28] Hindrance evasion LiDAR ATESOA algorithm [29] Hindrance evasion Microwave radar, 4 G camera - [29] Water quality monitoring Chl-aturbidity, water dissolved oxygen, water conductivity, oxidation-reduction potential, water temperature, salinity, Ph sensors - [30] Hindrance avoidance Laser scanner. Sonar, Alimeter VFH + algorithm [30] Water eminence Observation Chemical sensors (Hg, Cr, Cd and dispersed oil) - [31] Water eminence Observation Ph sensors, ORP sensor, salinity sensor, dissolved oxygen prob - [32] Water eminence Observation Ph sensor, TDS sensor, turbidity sensor Ensemble learning algorithm [33] Water eminence Observation CTD, DO, Ph, Pco2, nitrate, chlorophyll fluorance, CDOM, turbidity, DOC sensors - [34] Water surface cleaning Vision module, Grasper YOLOv3 algorithm [35] Water surface cleaning Conveyor belt - [36] Water surface cleaning Camera, waste scooper - [37] Water surface cleaning Camera, collection box YOLO v3 algorithm [38] Hindrance avoidance Ultrasonic sensors Threshold-based obstacle avoidance algorithm [38] Water quality monitoring Ph sensor - [38] Water surface cleaning Vision sensor, salvage net Hue-based colour filtering algorithm depth, oxygen dissolved, nitrate, colour dissolved organic matter, and turbidity. In this way, the autonomous cars increase the scope of their time-and space-based surveillance. ...
... Other works attempt to solve more complex problems, such as object detection and segmentation. In [11], the author uses two Neural Networks (NNs) to eliminate the ground and classify six kinds of waste items. We, however, use a single NN to locate and classify the objects, thus speeding up the process. ...
The accumulation of litter is increasing in many places and is consequently becoming a problem that must be dealt with. In this paper, we present a manipulator robotic system to collect litter in outdoor environments. This system has three functionalities. Firstly, it uses colour images to detect and recognise litter comprising different materials. Secondly, depth data are combined with pixels of waste objects to compute a 3D location and segment three-dimensional point clouds of the litter items in the scene. The grasp in 3 Degrees of Freedom (DoFs) is then estimated for a robot arm with a gripper for the segmented cloud of each instance of waste. Finally, two tactile-based algorithms are implemented and then employed in order to provide the gripper with a sense of touch. This work uses two low-cost visual-based tactile sensors at the fingertips. One of them addresses the detection of contact (which is obtained from tactile images) between the gripper and solid waste, while another has been designed to detect slippage in order to prevent the objects grasped from falling. Our proposal was successfully tested by carrying out extensive experimentation with different objects varying in size, texture, geometry and materials in different outdoor environments (a tiled pavement, a surface of stone/soil, and grass). Our system achieved an average score of 94% for the detection and Collection Success Rate (CSR) as regards its overall performance, and of 80% for the collection of items of litter at the first attempt.
... e Visual fault tolerance method (Yu et al., 2020). f Comparison of accuracy (Agrawal et al., 2010;Astrand & Baerveldt, 2002;Bai et al., 2018;Lu et al., 2018) background features. For example, in an orchard environment, the banana stalk, fruit, branches, and leaves are very similar in color. ...
... And specialized training for every possible feature is not needed. A novel garbage pickup robot is able to accurately and autonomously detect the garbage and segment the ground by using a deep neural network (Bai et al., 2018). However, deep learning method requires high-performance computing and large run-time memory footprint, which brings high computation overhead and power consumption. ...
... In sanitation, a novel garbage pickup robot is presented. Based on a powerful deep neural network, the robot can detect and pick up the garbage on the grass without any human help (Bai et al., 2018). Another garbage picking robot is designed based on Raspberry Pi. ...
Robot technology is considered one of the most promising technologies to achieve intelligent production, with picking robots being the most common type. Picking robots are highly integrated mechatronic systems, which autonomously complete tasks including picking, carrying, and sorting. The application of picking robots enhances the efficiency of production across various environments. In this paper, a classification-design-optimization-application integrated framework of picking robots is addressed, contributing to theoretical research and application of picking robots. Classification of picking robot is established and analyzed considering the differences of overall form and end-effector to guide the development of research strategies and approaches of picking robot. Design of picking robot is described from different aspects of the target, structure, monitoring, and control design. Additionally, the commonly used optimization methods for picking robots, including structural parameters, kinematics and dynamics, and energy consumption, are discussed. Finally, the application of picking robots under different environments is expounded, and the challenges and prospects of picking robots are highlighted. This study could present theoretical support and application measures for picking robot study and technology development.
... Vision systems along with public datasets have been proposed for litter recognition in water bodies to address the increasing waste accumulations [19]- [21]. Construction and demolition waste sorting was performed by Wang et al. [22], while the system of Bai et al. [23] enabled their robot to detect garbage on grass before picking it. Recently, a large dataset for general waste segmentation has been published [24], which highlights the complexity of the task from a vision perspective due to the extremely high shape (e.g., due to damages) and color (e.g., due to dirt) unpredictability that end-of-use items can have. ...
The linear take-make-dispose paradigm at the foundations of our traditional economy is proving to be unsustainable due to waste pollution and material supply uncertainties. Hence, increasing the circularity of material flows is necessary. In this paper, we make a step towards circular healthcare by developing several vision systems targeting three main circular economy tasks: resources mapping and quantification, waste sorting, and disassembly. The performance of our systems demonstrates that representation-learning vision can improve the recovery chain, where autonomous systems are key enablers due to the contamination risks. We also published two fully-annotated datasets for image segmentation and for key-point tracking in disassembly operations of inhalers and glucose meters. The datasets and source code are publicly available.
... To the best of our knowledge, there has been no prior research on indoor small suctionable object classification for robot vacuum cleaners, and the majority of the existing literature focuses on outdoor trash or garbage classification [14][15][16][17][18][19][20]. In this section, I conduct a review of the state-of-the-art AI models employed for outdoor garbage classification. ...
... The authors of [15] introduce a novel robot system with the primary objective of automating the process of collecting litter and garbage from grassy areas. This innovative system leverages the power of deep learning techniques to enable the robot to identify and classify various types of objects commonly found as litter in outdoor environments. ...
... In comparison to previous studies mentioned in refs. [15][16][17][18], the three selected models achieved a comparable test accuracy while consuming less memory. ...
Robot vacuum cleaners have gained widespread popularity as household appliances. One significant challenge in enhancing their functionality is to identify and classify small indoor objects suitable for safe suctioning and recycling during cleaning operations. However, the current state of research faces several difficulties, including the lack of a comprehensive dataset, size variation, limited visual features, occlusion and clutter, varying lighting conditions, the need for real-time processing, and edge computing. In this paper, I address these challenges by investigating a lightweight AI model specifically tailored for robot vacuum cleaners. First, I assembled a diverse dataset containing 23,042 ground-view perspective images captured by robot vacuum cleaners. Then, I examined state-of-the-art AI models from the existing literature and carefully selected three high-performance models (Xception, DenseNet121, and MobileNet) as potential model candidates. Subsequently, I simplified these three selected models to reduce their computational complexity and overall size. To further compress the model size, I employed post-training weight quantization on these simplified models. In this way, our proposed lightweight AI model strikes a balance between object classification accuracy and computational complexity, enabling real-time processing on resource-constrained robot vacuum cleaner platforms. I thoroughly evaluated the performance of the proposed AI model on a diverse dataset, demonstrating its feasibility and practical applicability. The experimental results show that, with a small memory size budget of 0.7 MB, the best AI model is L-w Xception 1, with a width factor of 0.25, whose resultant object classification accuracy is 84.37%. When compared with the most accurate state-of-the-art model in the literature, this proposed model accomplished a remarkable memory size reduction of 350 times, while incurring only a slight decrease in classification accuracy, i.e., approximately 4.54%.
... There have been studies on autonomous garbage collection vehicles, and some companies have even developed commercial autonomous garbage collection vehicles (Evan, 2018). In addition to these works, there exist numerous works on autonomous vehicle design for garbage collection (Jha et al., 2019;Mayorga et al., 2019;Banu and Florence, 2022;Diyva and Latha, 2021;Pyo et al. 2022;Ying and Zhang, 2018;Kulshreshtha et al. 2022;Assis, Biju, Alisha, Dhanadas, Kuria, 2012;Bai, Lian, Liu, Wang and Liu, 2018;Huang et al., 2022). However, many of these vehicles are large and require GPS localization to collect waste in specific locations, on the other hand, there exist only a few works considering fire detection. ...
... In addition, recognition, lane recognition, route planning, vehicle manipulation, and abnormal situation detection were considered in that work. In (Bai, Lian, Liu, Wang, and Liu, 2018), deep neuronal networks are used for object detection. However, different than our work, object detection algorithms in that work are not real-time. ...
Autonomous vehicles are becoming increasingly popular in a variety of applications, including waste collection and fire detection. In this work, we present the design and implementation of an autonomous vehicle for these tasks in urban environments. The vehicle is equipped with sensors and control algorithms to navigate, detect and collect plastic bottle wastes, and detect fires in real-time. The system uses an off-the-shelf, small-sized, battery-operated vehicle, a simple conveyor belt, and a vision-based, computerized system. Machine learning (ML-) based vision tasks are implemented to direct the vehicle to waste locations and initiate the waste removal process. A fire detection and alarm system are also incorporated, using a camera and machine learning algorithms to detect flames automatically. The vehicle was tested in a simulated urban environment, and the results demonstrate its effectiveness in waste material collection and fire detection. The proposed system has the potential to improve the efficiency and safety of such tasks in urban areas.
... Therefore, it needs a lot of hardware support [18]. In addition, deep learning has been successfully applied in separation and classification of waste electrical and electronic equipment (WEEE) batteries [19], E-waste collection [20], construction solid waste classification [21], and automatic detection of waste in water [22]. The efficient and accurate detection of domestic waste will assist the intelligent development of waste treatment. ...
It is of great significance to identify all types of domestic garbage quickly and intelligently to improve people's quality of life. Based on the visual analysis of feature map changes in different neural networks, a Skip-YOLO model is proposed for real-life garbage detection, targeting the problem of recognizing garbage with similar features. First, the receptive field of the model is enlarged through the large-size convolution kernel which enhanced the shallow information of images. Second, the high-dimensional features of the garbage maps are extracted by dense convolutional blocks. The sensitivity of similar features in the same type of garbage increases by strengthening the sharing of shallow low semantics and deep high semantics information. Finally, multiscale high-dimensional feature maps are integrated and routed to the YOLO layer for predicting garbage type and location. The overall detection accuracy is increased by 22.5% and the average recall rate is increased by 18.6% comparing the experimental results with the YOLOv3 analysis. In qualitative comparison, it successfully detects domestic garbage in complex multi-scenes. In addition, this approach alleviates the overfitting problem of deep residual blocks. The application case of waste sorting production line is used to further highlight the model generalization performance of the method.
