CN117755686B - A multi-task intelligent waste sorting device based on deep learning and mobile robotic grippers - Google Patents

A multi-task intelligent waste sorting device based on deep learning and mobile robotic grippers

Info

Publication number
CN117755686B
CN117755686B CN202410066054.1A CN202410066054A CN117755686B CN 117755686 B CN117755686 B CN 117755686B CN 202410066054 A CN202410066054 A CN 202410066054A CN 117755686 B CN117755686 B CN 117755686B
Authority
CN
China
Prior art keywords
waste
garbage
detection
controller
detection platform
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410066054.1A
Other languages
Chinese (zh)
Other versions
CN117755686A (en
Inventor
吴燕燕
包梓含
徐广宇
李威翰
张家宁
陈发祥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenyang Aerospace University
Original Assignee
Shenyang Aerospace University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenyang Aerospace University filed Critical Shenyang Aerospace University
Priority to CN202410066054.1A priority Critical patent/CN117755686B/en
Publication of CN117755686A publication Critical patent/CN117755686A/en
Application granted granted Critical
Publication of CN117755686B publication Critical patent/CN117755686B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02WCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO WASTEWATER TREATMENT OR WASTE MANAGEMENT
    • Y02W30/00Technologies for solid waste management
    • Y02W30/10Waste collection, transportation, transfer or storage, e.g. segregated refuse collecting, electric or hybrid propulsion

Landscapes

  • Image Analysis (AREA)

Abstract

本发明提供一种基于深度学习和移动机械爪的多任务垃圾智能分拣装置,涉及垃圾分类技术领域。该装置包括顶端设置垃圾投放口的分拣箱箱体以及设置在箱体内部的垃圾分拣设备、摄像头、开发板和控制器;摄像头实时检测通过垃圾投放口投入的垃圾图像并传输给开发板,开发板通过视觉算法识别并给出相应垃圾类别和坐标位置,并传输至控制器;控制器根据接收到的垃圾坐标值控制垃圾分拣设备投入箱体下方对应的垃圾桶内。该装置通过软硬件结合,并加入目标定位,配合机械手爪,解决了多种类垃圾的分类识别和投递的问题。

This invention provides a multi-task intelligent waste sorting device based on deep learning and a mobile robotic gripper, relating to the field of waste sorting technology. The device includes a sorting bin with a waste inlet at the top, and waste sorting equipment, a camera, a development board, and a controller housed inside the bin. The camera detects images of waste entering through the inlet in real time and transmits them to the development board. The development board uses a visual algorithm to identify the waste type and coordinates, transmitting this information to the controller. The controller then uses the received coordinates to control the waste sorting equipment to place the waste into the corresponding bin below the bin. This device, through a combination of hardware and software, incorporating target localization and a robotic gripper, solves the problem of classifying, identifying, and disposing of various types of waste.

Description

Multi-task garbage intelligent sorting device based on deep learning and movable mechanical claws
Technical Field
The invention relates to the technical field of garbage classification, in particular to a multi-task intelligent garbage sorting device based on deep learning and mobile mechanical claws.
Background
Today, environmental problems develop gradually, environmental pollution is aggravated gradually, and global attention is attracted. Improper disposal of waste is one of the main factors causing environmental pollution, and ordered waste classification becomes the primary task to solve such pollution. In recent years, china is gradually pushing garbage classification all over the country, good effects are shown, 46 main cities are tested in the early stage, and positive development of garbage classification is promoted.
The garbage classification can help us maximize the utilization of garbage resources, reduce the total amount of garbage, improve our living environment, reduce the pollution of garbage to underground water, and reduce the occupied area of garbage. Many waste contains substances that are difficult to decompose naturally, which are very harmful to the land, while waste classification enables the identification and individual disposal of these recoverable or difficult to degrade substances, thus reducing the amount of waste by at least 60% or more.
Therefore, garbage classification is particularly important as a primary link in garbage disposal flow, and intelligent garbage classification becomes a research hotspot. Traditional intelligent garbage classification device mostly can satisfy rubbish and drops into one by one, then through the technique of computer vision detection, accomplishes the autonomic judgement and the classified storage to various rubbish, to the condition that a plurality of different kinds of rubbish were mixed together drops into the garbage bin, traditional intelligent garbage bin's design can not carry out intelligent categorised delivery and storage, has influenced holistic letter sorting efficiency.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides the intelligent multi-task garbage sorting device based on deep learning and mobile mechanical claws, which realizes sorting delivery and storage of a plurality of different types of garbage.
The intelligent multi-task garbage sorting device comprises a sorting box body, garbage sorting equipment, a camera, a development board and a controller, wherein the top end of the sorting box body is provided with a garbage throwing port, the garbage sorting equipment, the camera, the development board and the controller are arranged in the box body, the camera detects garbage images thrown in through the garbage throwing port in real time and transmits the garbage images to the development board, the development board recognizes and gives corresponding garbage types and coordinate positions through a visual algorithm and transmits the garbage types and coordinate positions to the controller, and the controller controls the garbage sorting equipment to throw in garbage cans corresponding to the lower portion of the box body according to received garbage coordinate values.
Preferably, the device further comprises a detection platform and a tray arranged in the sorting box body; the detection platform is positioned at the lower part of the garbage throwing opening and consists of an acrylic plate with an opening and a closing middle, two ends of the acrylic plate are respectively fixed on an optical axis, the optical axis is controlled to rotate through a first steering engine to realize the opening and the closing of the detection platform, the tray is arranged below the detection platform and controls the rotating angle through a second steering engine, and the first steering engine and the second steering engine control the rotating angle through a controller.
The STM32 controller drives the stepping motor to control the two-dimensional sliding table and the mechanical claw to move to the coordinate position of the garbage when receiving the coordinate value of the current garbage, starts the mechanical claw to grab the garbage and returns to the central position of the detection platform, at the moment, the first steering engine for controlling the opening and closing of the detection platform rotates for 90 degrees, the detection platform is opened, the garbage falls into a tray controlled by a second steering engine below, and the second steering engine controls the tray to rotate to the position corresponding to the garbage can, so that the garbage can be accurately classified.
Preferably, the device further comprises a display screen arranged at the top of the sorting box body and used for displaying sorting and delivering results of various garbage in real time.
Preferably, the development board deploys a visual algorithm to calculate the object type and specific coordinate value of the identified garbage, and feeds back the coordinate value to the controller, and the controller controls the stepping motor to realize the movement of the two-dimensional sliding table, so that the mechanical gripper is positioned to the garbage with the specific coordinate value, and the garbage is grabbed.
Preferably, the specific method for calculating the specific coordinate value of the identified garbage by using the development board deployment visual algorithm comprises the following steps:
step 1, constructing a garbage classification detection model based on YOLOv and a normalized attention module;
Adopting YOLOv network model and normalized attention module as main frame of garbage classification detection model, wherein YOLOv network model is divided into 4 parts, namely main feature extraction network, feature fusion network, DBL convolution layer and loss function, wherein in YOLOv network, a normalized attention module is embedded at the end of each network block to make the network lighter and improve detection efficiency;
Step2, counting common household garbage types and constructing a training data set training garbage classification detection model;
in order to realize the accurate classification of multiple types of garbage, a multi-classification model is trained, and the accurate classification of the multiple types of garbage is realized;
step 3, identifying and classifying the garbage put on the detection platform by using a trained garbage classification detection model, calculating the coordinate position of the garbage put on the detection platform and marking the coordinate position;
and 4, returning the relative coordinates of the targets within the detection range of the camera by utilizing the target detection characteristics of the YOLO model, acquiring depth information by measuring the distance between the camera and the detection platform, mapping the coordinates of the targets acquired by the camera to the coordinates of the detection targets in the world coordinate system by vector transformation, and converting the coordinates of the detection targets in the world coordinate system into the coordinates of the targets of the two-dimensional sliding table by proportional operation, so as to realize accurate grabbing of the target garbage.
Preferably, the development board sends the identified garbage category and the coordinate position to the controller through a serial port protocol, the controller drives the stepping motor to move by using an acceleration and deceleration algorithm after receiving corresponding information and moves to the coordinate position of the corresponding garbage, the controller drives the steering engine to drive the mechanical gripper to move, the target garbage is grabbed, the target garbage returns to the middle position of the detection platform after being grabbed, the garbage is dropped and the detection platform is started, and the garbage falls into a tray controlled by the steering engine, so that the garbage is put into the specified garbage can.
Preferably, the device further comprises a diffuse reflection photoelectric switch for detecting whether the garbage can is full, the diffuse reflection photoelectric switch is controlled by the controller, the photoelectric switch is arranged at the position of each garbage can at a fixed height, when garbage in the garbage can reaches the height, the voice broadcasting module is started to carry out full-load prompt, corresponding garbage can full-load display is given on the display screen, and further full-load judgment of the garbage can is realized.
The technical proposal has the advantages that the intelligent multi-task garbage sorting device based on the deep learning and the moving mechanical claw,
(1) By using the improved YOLOv5+NAM attention module, the network is lighter, the detection efficiency and the anti-interference capability are further improved, and the detection robustness and accuracy are higher;
(2) An electric control algorithm is improved, and an acceleration and deceleration algorithm is used for driving a stepping motor, so that the garbage throwing work after clamping is more stable;
(3) The problems of classification, identification and delivery of various garbage are solved by combining software and hardware, adding target positioning and matching with a mechanical gripper.
Drawings
Fig. 1 is a schematic structural diagram of a multi-task intelligent garbage sorting device based on deep learning and moving mechanical claws, which is provided by the embodiment of the invention;
fig. 2 is a flow chart of intelligent garbage sorting by the multi-task intelligent garbage sorting device based on deep learning and moving mechanical claws, which is provided by the embodiment of the invention;
Fig. 3 is a schematic diagram of diffuse reflection photoelectric switch detection provided in an embodiment of the present invention, where (a) is a schematic diagram of diffuse reflection switch installation, and (b) is a schematic diagram of full load detection;
FIG. 4 is a diagram illustrating a YOLOV5+NAM model architecture for an improvement provided by an embodiment of the present invention;
FIG. 5 is a schematic diagram of a CBAM-channel attention sub-module provided by an embodiment of the present invention;
FIG. 6 is a schematic diagram of a SAM space attention sub-module according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a network model YOLOV of an improved portion according to an embodiment of the present invention;
FIG. 8 is a diagram of automatic labeling results of targets on an original image according to an embodiment of the present invention;
fig. 9 is a schematic diagram of a back-deduced relative position coordinate according to an embodiment of the present invention.
In the figure, 1, a camera; 2, a highlight display screen, 3, a mechanical gripper, 4, a second steering engine, 5, a first steering engine, 6, a garbage can, 7, an acrylic plate, 8, a stepping motor, 9, an aluminum profile, 10, a two-dimensional sliding table, 11 and a detection platform.
Detailed Description
The following describes in further detail the embodiments of the present invention with reference to the drawings and examples. The following examples are illustrative of the invention and are not intended to limit the scope of the invention.
In the embodiment, the intelligent multi-task garbage sorting device based on deep learning and moving mechanical claws is shown in fig. 1 and 2, and comprises a sorting box body with a garbage throwing port at the top end, a detection platform 11, a tray, a first steering engine 5, a second steering engine 4, garbage sorting equipment, cameras 1, jetson nano development boards and an STM32 controller, wherein the detection platform 11, the cameras 1 detect garbage images thrown through the garbage throwing port in real time and transmit the garbage images to the jetson nano development boards, the jetson nano development boards recognize and give out coordinate positions of corresponding garbage through visual algorithms, the recognized garbage types and the coordinate positions are sent to the STM32 controller through serial port protocols, the detection platform 11 is positioned at the lower part of the garbage throwing port and is composed of an acrylic plate 7 with an opening and closing middle, two ends of the acrylic plate 7 are respectively fixed on an optical axis, the optical axis is controlled to rotate through the first steering engine 5, the opening and closing of the detection platform 11 are realized, the tray is arranged below the detection platform 11, the rotation angle is controlled through the second steering engine 4, and the first steering engine 5 and the second steering engine 4 control the rotation angle is controlled through the controller.
The garbage sorting equipment comprises a two-dimensional sliding table 10, a flexible mechanical claw 3 and a stepping motor 8, when the STM32 controller receives the coordinate value of the current garbage, the stepping motor 8 is driven by an acceleration and deceleration algorithm to control the two-dimensional sliding table 10 and the mechanical claw 3 to move to the coordinate position of the garbage, the mechanical claw 3 is started to grab garbage and return to the central position of the detection platform 11, at the moment, the detection platform 11 is controlled to be opened and closed by a first steering engine 5 to rotate for 90 degrees, the detection platform 11 is opened (only an acrylic plate on one side is controlled to be opened), the garbage falls into a tray controlled by a second steering engine 4 below, and the second steering engine 4 controls the tray to rotate to the position corresponding to the garbage can, so that the garbage can be accurately sorted.
Meanwhile, the device also comprises a highlight display screen 2 arranged at the top of the sorting box body and diffuse reflection photoelectric switches arranged on the inner walls of the garbage cans, as shown in fig. 3, wherein the display screen 2 is used for displaying sorting and delivering results of various garbage in real time. The diffuse reflection photoelectric switch is controlled by the STM32 singlechip to detect whether the dustbin is full. In this embodiment, this photoelectric switch is installed in four garbage bins high 80%'s position, and when the height of rubbish reaches 80% in having the garbage bin, start voice broadcast module and carry out full load suggestion to give corresponding garbage bin full load display on display screen 2, and then judge whether there is the garbage bin full load.
In this embodiment, the letter sorting case box adopts 4040 aluminium alloy 9 as the frame, and the link between the aluminium alloy 9 adopts M5 bolt and ship type nut to fasten, and the device is all around and the top adopts black acrylic board to encapsulate, and a high-brightness display screen 2 is placed at the box top and is used for showing the real-time classification and the delivery result of all kinds of rubbish, designs 200 at the top of box and puts in mouthful for the delivery of multitasking rubbish. When a plurality of garbage is input into the detection platform 11 through the input port, the camera 1 at the top of the box detects the garbage, the jetson nano development board accurately identifies the garbage type through a visual algorithm and gives the coordinate position of the corresponding garbage, the STM32 controller drives the stepping motor 8 to control the two-dimensional sliding table 10 and the mechanical claw 3 to move to the coordinate position of the garbage by using an acceleration-deceleration algorithm when receiving the coordinate value of the current garbage, the mechanical claw 3 is started to grab, the garbage is grabbed and returned to the central position of the detection platform 10, the first steering engine 5 controlling the opening and closing of the detection platform 10 is rotated for 90 degrees at the moment, the detection platform 11 is opened, the garbage falls into a tray controlled by the lower steering engine, so that the garbage falls into the corresponding garbage can, and if the camera 1 detects the garbage on the platform 11, the mechanical claw 3 returns to the coordinate position of the next garbage, grabs and inputs the garbage into the corresponding garbage can until the garbage on the detection platform is sorted, and intelligent classification and delivery of the multi-task garbage are realized. In the embodiment, four sub-garbage cans 6 with the size of 200mm long by 200mm wide by 350mm high are designed at the bottom of the box body and are respectively used for storing recyclable garbage, kitchen garbage, harmful garbage and other garbage, diffuse reflection photoelectric switches are arranged at the positions of 80% of the four garbage cans, so that full load detection of the garbage cans is realized, and when the garbage cans are full load, corresponding voice prompts are given and corresponding information such as full load of the garbage cans is displayed on a screen;
In the embodiment, jetson nano develops a board deployment visual algorithm to calculate the object type and specific coordinate value of the identified garbage, and feeds back the coordinate value to the STM32 controller, and the controller controls the stepping motor to realize the movement of the two-dimensional sliding table, so that the mechanical gripper is positioned to the garbage with the specific coordinate value, and the garbage is grabbed.
The concrete method for calculating the concrete coordinate value of the identified garbage by using jetson nano development board deployment visual algorithm comprises the following steps:
Step 1, constructing a garbage classification detection model based on YOLOv and a Normalized Attention Module (NAM) (Normalization-based Attention Module);
adopting YOLOv network model and normalized attention module NAM as main frame of garbage classification detection model, as shown in figure 4, YOLOv network model is divided into 4 parts, respectively extracting network, feature fusion network, DBL convolution layer and loss function for main feature, embedding NAM module at the end of each network block in YOLOv network, making the network lighter and improving detection efficiency;
NAM is an efficient and lightweight attention module, which directly uses scaling factors of batch normalization (Batch Normalization, BN) to calculate attention weights without requiring additional calculations and parameters such as full connection, convolution, etc., and further suppresses insignificant features by adding regularization terms, as compared to conventional attention modules. NAM adopts channel attention module (CBAM) and Space Attention Module (SAM) to redesign channel attention and space attention sub-module, in CBAM, BN is used to measure the importance of pixels, namely pixel normalization, as shown in formula (1), scaling factors in BN reflect the change of each channel and also indicate the importance of the channel, scaling factors, namely variances in BN, the larger the variances indicate the more varied the channel, the more abundant the information contained in the channel, the larger the importance, and the channels with little change, the single information and small importance.
Where μ B and σ B are the mean and standard deviation, respectively, of small lot B, and γ and β are trainable affine transformation parameters (scale and displacement).
The CBAM-channel attention sub-module is shown in formula (2) and fig. 5, where M c represents the output characteristics, sigmoid represents the activation function, γ is the scale factor of each channel, and the weight W γ=γi/∑j=0γj,F1 represents the input characteristics. If the same normalization method is used for each pixel in space, the weight of the spatial attention, i.e. pixel normalization, can be obtained;
The SAM spatial attention sub-module is shown in equation (3) and fig. 6, M s represents the output feature, sigmoid represents the activation function, ρ is the scale factor, weight W ρ=ρi/∑j=0ρj,F2 represents the input feature,
Mc=sigmoid(Wγ(BN(F1))) (2)
Ms=sigmoid(Wρ(BNs(F2)) (3)
The NAM attention module adds a regularization term to its loss function, shown in equation (4), to suppress unimportant features.
Loss=∑(x,y)l(f(x,W),y)+p∑g(γ)+p∑g(ρ) (4)
In the above formula, x is input, y is output, W is the network weight, f (·) is the input feature, l (·) is the loss function, g (·) is the l 1 norm penalty function, and p is the penalty parameter for balancing g (γ) and g (ρ).
The invention combines YOLOv with NAM module to obtain improved YOLOv network model as shown in figure 7 as garbage classification detection model;
Step2, counting common household garbage types and constructing a training data set training garbage classification detection model;
In order to realize accurate classification of multiple types of garbage, the embodiment trains a multi-classification model, can realize accurate classification of multiple types of garbage, and can identify plastic bottles, paper cups, pop cans and the like in common garbage.
Step 3, identifying and classifying the garbage put on the detection platform by using a trained garbage classification detection model, classifying the common garbage into four types of recyclable garbage, kitchen garbage, harmful garbage and other garbage in the detection process, calculating the coordinate position of the garbage put on the detection platform, and marking the coordinate position as shown in fig. 8;
step 4, returning to the relative coordinates of the target in the detection range of the camera by utilizing the target detection characteristic of the YOLO model, obtaining depth information by measuring the distance between the camera and the detection platform, mapping the target coordinates obtained by the camera to the coordinates of the detection target in the world coordinate system by vector conversion, and converting the coordinates of the detection target in the world coordinate system into the target coordinates of the two-dimensional slipway by simple proportional operation because the two-dimensional slipway has no influence of three-dimensional parameters, wherein the accurate grabbing of the target garbage can be realized by the algorithm as shown in fig. 9;
in this embodiment, the upper left corner of the detection platform is taken as the origin of the coordinate system, the detect. Py file is found in the algorithm code, and the plot_one_box is opened and found, and the code is as follows:
then, ctr+mouse clicks, enters into general. Py, and automatically locates to the plot_one_box function, and modifies the function into
And outputting the target coordinate information to obtain the coordinate position of the garbage to be grabbed.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will understand that they can make modifications to the technical solutions described in the above-mentioned embodiments or make equivalent substitutions of some or all of the technical features, without departing from the essence of the corresponding technical solutions from the scope of the invention defined by the claims.

Claims (4)

1.一种基于深度学习和移动机械爪的多任务垃圾智能分拣装置,其特征在于:包括顶端设置垃圾投放口的分拣箱箱体以及设置在箱体内部的垃圾分拣设备、摄像头、开发板和控制器;所述摄像头实时检测通过垃圾投放口投入的垃圾图像并传输给开发板,开发板通过视觉算法识别并给出相应垃圾类别和坐标位置,并传输至控制器;控制器根据接收到的垃圾坐标值控制垃圾分拣设备投入箱体下方对应的垃圾桶内;1. A multi-task intelligent waste sorting device based on deep learning and a mobile robotic gripper, characterized in that: it includes a sorting box body with a waste inlet at the top, and waste sorting equipment, a camera, a development board, and a controller disposed inside the box body; the camera detects images of waste entering through the waste inlet in real time and transmits them to the development board, the development board identifies the waste type and coordinate position using a visual algorithm, and transmits this information to the controller; the controller controls the waste sorting equipment to put the waste into the corresponding waste bin below the box body based on the received waste coordinate values; 所述装置还包括设置在分拣箱箱体内的检测平台和托盘;所述检测平台位于垃圾投放口下部,由中间开合的亚克力板组成,亚克力板两端分别固定在光轴上,通过第一舵机控制光轴转动,实现检测平台开合;所述托盘设置在检测平台下方,并通过第二舵机控制旋转角度;所述第一舵机和第二舵机通过控制器控制旋转角度;The device also includes a detection platform and a tray installed inside the sorting box; the detection platform is located below the waste disposal opening and is composed of an acrylic plate that opens and closes in the middle, with both ends of the acrylic plate fixed to an optical axis. The optical axis is rotated by a first servo motor to realize the opening and closing of the detection platform; the tray is installed below the detection platform and its rotation angle is controlled by a second servo motor; the first and second servo motors are controlled by a controller to control the rotation angle. 所述垃圾分拣设备包括二维滑台、机械手爪、步进电机;STM32控制器接收到当前垃圾的坐标值时,使用加减速算法驱动步进电机控制二维滑台和机械手爪运动到该垃圾的坐标位置,启动机械手爪进行抓取,抓取到垃圾并返回检测平台中央位置,此时控制检测平台开合的第一舵机旋转90度,检测平台打开,垃圾掉入下面第二舵机控制的托盘,第二舵机控制托盘旋转到对应垃圾桶的位置,实现垃圾的准确分类;The waste sorting equipment includes a two-dimensional slide, a robotic gripper, and a stepper motor. When the STM32 controller receives the coordinate value of the current waste, it uses an acceleration and deceleration algorithm to drive the stepper motor to control the two-dimensional slide and the robotic gripper to move to the coordinate position of the waste. The robotic gripper is then activated to grab the waste and returns to the center of the detection platform. At this time, the first servo motor that controls the opening and closing of the detection platform rotates 90 degrees, the detection platform opens, and the waste falls into the tray controlled by the second servo motor below. The second servo motor controls the tray to rotate to the position of the corresponding waste bin, thus achieving accurate waste sorting. 所述开发板部署视觉算法推算出所识别垃圾的物体类别和具体坐标值,并将此坐标值反馈给控制器,控制器控制步进电机实现二维滑台的运动,进而将机械手爪定位到具体坐标值的垃圾,实现垃圾的抓取;The development board deploys a visual algorithm to calculate the object category and specific coordinate value of the identified waste, and feeds this coordinate value back to the controller. The controller controls the stepper motor to realize the movement of the two-dimensional slide, thereby positioning the robotic gripper to the waste with specific coordinate values and realizing the grabbing of the waste. 所述开发板部署视觉算法推算出所识别垃圾的具体坐标值的具体方法为:The specific method by which the visual algorithm deployed on the development board calculates the specific coordinates of the identified waste is as follows: 步骤1、基于YOLOv5和规范化的注意力模块构建垃圾分类检测模型;Step 1: Build a garbage classification detection model based on YOLOv5 and a normalized attention module; 采用YOLOv5网络模型+规范化的注意力模块作为垃圾分类检测模型的主框架,所述YOLOv5网络模型分为4个部分,分别为主干特征提取网络、特征融合网络、DBL卷积层和损失函数;在YOLOv5网络中,每个网络块的末尾嵌入一个规范化的注意力模块,使网络更加轻量化并且提高检测的效率;The YOLOv5 network model with a normalized attention module is used as the main framework of the garbage classification detection model. The YOLOv5 network model is divided into four parts: backbone feature extraction network, feature fusion network, DBL convolutional layer and loss function. In the YOLOv5 network, a normalized attention module is embedded at the end of each network block, which makes the network more lightweight and improves the detection efficiency. 步骤2、统计常见的生活垃圾种类并构建训练数据集训练垃圾分类检测模型;Step 2: Collect statistics on common types of household waste and build a training dataset to train a waste classification detection model; 在检测环境及非检测环境中分别采集多张不同种类的垃圾图片,并将其标注构建训练数据集;为了实现多种类型垃圾的精确分类,训练了一个多分类模型,实现多种类型垃圾的精确分类;Multiple images of different types of waste were collected in both detection and non-detection environments and labeled to construct a training dataset. In order to achieve accurate classification of multiple types of waste, a multi-classification model was trained to achieve accurate classification of multiple types of waste. 步骤3、使用训练好的垃圾分类检测模型对投入到检测平台上的垃圾进行识别分类,并计算出投入到检测平台上垃圾的坐标位置并对其进行标框;Step 3: Use the trained waste sorting and detection model to identify and classify the waste put into the detection platform, calculate the coordinates of the waste put into the detection platform, and mark it with a bounding box. 步骤4、利用YOLO模型的目标检测特性,返回目标在摄像头检测范围内的相对坐标,通过测量摄像头和检测平台间的距离来获取深度信息,再通过向量变换将摄像头获取的目标坐标映射到检测目标在世界坐标系中的坐标,通过比例运算将检测目标在世界坐标系中的坐标转换为二维滑台的目标坐标,实现对目标垃圾的精准抓取。Step 4: Utilize the target detection characteristics of the YOLO model to return the relative coordinates of the target within the camera's detection range. Obtain depth information by measuring the distance between the camera and the detection platform. Then, map the target coordinates obtained by the camera to the target's coordinates in the world coordinate system through vector transformation. Finally, convert the target's coordinates in the world coordinate system into the target coordinates of the two-dimensional sliding table through scaling operations to achieve precise grabbing of target debris. 2.根据权利要求1所述的基于深度学习和移动机械爪的多任务垃圾智能分拣装置,其特征在于:所述装置还包括设置在分拣箱箱体顶部的显示屏,用于实时显示各类垃圾的分类和投递结果。2. The multi-task intelligent waste sorting device based on deep learning and mobile robotic claw according to claim 1, characterized in that: the device further includes a display screen installed on the top of the sorting box for real-time display of the classification and delivery results of various types of waste. 3.根据权利要求2所述的基于深度学习和移动机械爪的多任务垃圾智能分拣装置,其特征在于:所述开发板通过串口协议将识别出的垃圾类别及坐标位置发送给控制器,控制器接收到对应信息后使用加减速算法驱动步进电机运动,移动到相应垃圾的坐标位置,控制器驱动舵机带动机械手爪运动,对目标垃圾进行抓取,抓取到垃圾之后返回检测平台中间位置,将垃圾落下同时检测平台开启,垃圾落入舵机控制的托盘,从而实现将垃圾投放到指定垃圾桶。3. The multi-task intelligent waste sorting device based on deep learning and a mobile robotic gripper as described in claim 2, characterized in that: the development board sends the identified waste type and coordinate position to the controller via a serial port protocol; after receiving the corresponding information, the controller uses an acceleration/deceleration algorithm to drive the stepper motor to move to the coordinate position of the corresponding waste; the controller drives the servo motor to move the robotic gripper to grab the target waste; after grabbing the waste, it returns to the middle position of the detection platform, drops the waste, and simultaneously opens the detection platform, allowing the waste to fall into the tray controlled by the servo motor, thereby realizing the disposal of waste into the designated waste bin. 4.根据权利要求3所述的基于深度学习和移动机械爪的多任务垃圾智能分拣装置,其特征在于:所述装置还包括漫反射光电开关来检测该垃圾桶是否满载,通过控制器控制漫反射光电开关,该光电开关被安装在各垃圾桶固定高度的位置,当有垃圾桶内垃圾达到该高度时,进行语音满载提示,并在显示屏上给出相应垃圾桶满载显示,进而实现垃圾桶满载判断。4. The multi-task intelligent waste sorting device based on deep learning and mobile mechanical claw according to claim 3, characterized in that: the device further includes a diffuse reflection photoelectric switch to detect whether the waste bin is full, and the diffuse reflection photoelectric switch is controlled by the controller. The photoelectric switch is installed at a fixed height of each waste bin. When the waste in a waste bin reaches the height, a voice prompt indicating that the waste bin is full is given, and the corresponding waste bin full is displayed on the screen, thereby realizing the judgment of the waste bin being full.
CN202410066054.1A 2024-01-17 2024-01-17 A multi-task intelligent waste sorting device based on deep learning and mobile robotic grippers Active CN117755686B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410066054.1A CN117755686B (en) 2024-01-17 2024-01-17 A multi-task intelligent waste sorting device based on deep learning and mobile robotic grippers

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410066054.1A CN117755686B (en) 2024-01-17 2024-01-17 A multi-task intelligent waste sorting device based on deep learning and mobile robotic grippers

Publications (2)

Publication Number Publication Date
CN117755686A CN117755686A (en) 2024-03-26
CN117755686B true CN117755686B (en) 2025-12-12

Family

ID=90325867

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410066054.1A Active CN117755686B (en) 2024-01-17 2024-01-17 A multi-task intelligent waste sorting device based on deep learning and mobile robotic grippers

Country Status (1)

Country Link
CN (1) CN117755686B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113277239A (en) * 2021-05-25 2021-08-20 江汉大学 Garbage classification equipment
CN113562355A (en) * 2021-08-10 2021-10-29 南京航空航天大学 Intelligent garbage sorting device and method based on deep learning technology
CN215853160U (en) * 2021-08-10 2022-02-18 西安电子科技大学 Intelligent garbage can
CN116758416A (en) * 2023-05-29 2023-09-15 浙江大学 An industrial garbage target detection method based on improved YOLOv5 and model transfer learning fine-tuning

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102012561B1 (en) * 2017-11-16 2019-08-20 성민준 Automatic separation garbage can using sound recognition
CN214826172U (en) * 2021-03-18 2021-11-23 西安科技大学 An intelligent sorting trash can based on visual servo system
CN113128363A (en) * 2021-03-31 2021-07-16 武汉理工大学 Machine vision-based household garbage sorting system and method
CN218490505U (en) * 2022-03-10 2023-02-17 桂林理工大学 Intelligent classification dustbin
CN117262524A (en) * 2023-09-14 2023-12-22 黄河科技学院 Garbage self-classifying method based on deep learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113277239A (en) * 2021-05-25 2021-08-20 江汉大学 Garbage classification equipment
CN113562355A (en) * 2021-08-10 2021-10-29 南京航空航天大学 Intelligent garbage sorting device and method based on deep learning technology
CN215853160U (en) * 2021-08-10 2022-02-18 西安电子科技大学 Intelligent garbage can
CN116758416A (en) * 2023-05-29 2023-09-15 浙江大学 An industrial garbage target detection method based on improved YOLOv5 and model transfer learning fine-tuning

Also Published As

Publication number Publication date
CN117755686A (en) 2024-03-26

Similar Documents

Publication Publication Date Title
US20250353670A1 (en) Systems and methods for detecting waste receptacles
CN109928107B (en) An Automatic Classification System
CN110924340B (en) Mobile robot system for intelligently picking up garbage and implementation method
CN111439500B (en) Garbage automatic classification method and garbage automatic classification device
US9403278B1 (en) Systems and methods for detecting and picking up a waste receptacle
CN110654739A (en) Machine vision-based automatic recyclable garbage classification recycling device and method
CN110001875A (en) It is a kind of to be cleaned and the water surface robot classified and Refloatation method of classifying for garbage on water
CN110937280B (en) Audio-visual combination based intelligent garbage classification and recovery method and terminal
CN110589282A (en) Intelligent garbage classification method based on machine learning and automatic garbage sorting device
CN111583671B (en) A millimeter wave radar intersection traffic flow monitoring method and system
CN114955289A (en) Intelligent garbage sorting, recycling and management method and intelligent sorting trash can
CN118385157A (en) Visual classified garbage automatic sorting system based on deep learning and self-adaptive grabbing
CN115185280A (en) Unmanned ship control system and method based on path planning autonomous navigation
CN215853160U (en) Intelligent garbage can
CN117755686B (en) A multi-task intelligent waste sorting device based on deep learning and mobile robotic grippers
CN114506591A (en) A design method of intelligent trash can based on image classification technology
CN111498326A (en) Automatic garbage classifier based on two-class recognition model
CN119472658A (en) Unmanned vessel cluster surface partitioning collaborative classification salvage method based on deep learning
CN116664871B (en) An intelligent control method and system based on deep learning
CN212607180U (en) Triangular intelligent garbage can
CN111731715A (en) A smart trash can that can actively sort garbage
CN118239142B (en) A waste sorting and processing device and method
CN222203636U (en) Garbage classification processing device
CN212830606U (en) Automatic sorting trash can with detection function
CN121425686A (en) A smart sorting trash can based on image recognition

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant