CN110743818A - Garbage sorting system and garbage sorting method based on vision and deep learning - Google Patents
Garbage sorting system and garbage sorting method based on vision and deep learning Download PDFInfo
- Publication number
- CN110743818A CN110743818A CN201911196746.3A CN201911196746A CN110743818A CN 110743818 A CN110743818 A CN 110743818A CN 201911196746 A CN201911196746 A CN 201911196746A CN 110743818 A CN110743818 A CN 110743818A
- Authority
- CN
- China
- Prior art keywords
- garbage
- deep learning
- target
- grabbing
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000010813 municipal solid waste Substances 0.000 title claims abstract description 216
- 238000013135 deep learning Methods 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 21
- 239000000463 material Substances 0.000 claims abstract description 29
- 238000012549 training Methods 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 6
- 238000004891 communication Methods 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 11
- 238000013136 deep learning model Methods 0.000 claims description 9
- 230000000007 visual effect Effects 0.000 claims description 6
- 238000007599 discharging Methods 0.000 claims description 5
- 230000001502 supplementing effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 239000011087 paperboard Substances 0.000 description 3
- 239000004033 plastic Substances 0.000 description 3
- 229920003023 plastic Polymers 0.000 description 3
- 239000002023 wood Substances 0.000 description 3
- 238000003912 environmental pollution Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 239000004575 stone Substances 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 238000005188 flotation Methods 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000003973 paint Substances 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/34—Sorting according to other particular properties
Abstract
The invention provides a garbage sorting system and a garbage sorting method based on vision and deep learning, wherein the garbage sorting system based on vision and deep learning comprises a conveying device; the image acquisition device comprises a linear array camera for acquiring 2D image data of the garbage on the conveying device and a 3D camera for acquiring 3D point cloud data of the garbage on the conveying device; the grabbing device is used for grabbing the target garbage; the control unit comprises an image processing module for registering the obtained 2D image data and 3D point cloud data to obtain RGB-D image data, and a deep learning training and calculating module for performing deep learning on the RGB-D image data to identify the material of the garbage to obtain target garbage and calculating the position information of the target garbage captured by the capturing device; and the accuracy rate and the recognition rate of target garbage recognition are enhanced.
Description
Technical Field
The invention relates to the field of garbage recovery, in particular to a garbage sorting system and a garbage sorting method based on vision and deep learning, which can automatically sort garbage.
Background
The problem of environmental pollution caused by garbage is increasingly serious, and particularly, with the acceleration of the urbanization speed of China, a large amount of building garbage such as concrete blocks, broken stone blocks, wood boards, paperboards, plastics, cloth art and the like are generated along with the removal and the demolition of a large amount of buildings in the process of modifying cities and towns, so that serious environmental pollution and resource waste are caused due to improper treatment.
At present, the sorting mode of garbage sorting mainly adopts mechanical sorting technologies such as size screening, air sorting, liquid flotation, bouncing sorting, magnetic sorting, eddy current sorting, sensor airflow sorting and the like to sort garbage, wherein manual sorting is an indispensable link in the sorting technologies, and the manual sorting is mainly used for selecting large-size target garbage which cannot be identified by the mechanical sorting technologies, such as plastics, wood boards, paper boards, large combustible materials, paint buckets and the like. However, manual sorting has the problems of low sorting efficiency, poor working environment, damage to the physical health of workers and the like.
At present, some domestic scientific research institutions try to build a test environment in a laboratory, screen target objects by using a visual technology and mostly depend on RGB images, but the high requirements on object recognition accuracy in modern industrial application cannot be met due to the limitation that the RGB images and gray level images contain information and the like, the industrial application cannot adapt to the complex use environment of a client site, and industrialization cannot be realized.
In view of the above, there is a need to provide a new visual and deep learning based garbage sorting system and method to solve the above problems.
Disclosure of Invention
The invention aims to provide a vision and deep learning based garbage sorting system and a garbage sorting method capable of automatically sorting garbage.
In order to achieve the purpose, the invention adopts the following technical scheme: a vision and deep learning based garbage sorting system comprises a conveying device used for conveying garbage to be sorted;
the image acquisition device comprises a linear array camera for acquiring 2D image data of the garbage on the conveying device and a 3D camera for acquiring 3D point cloud data of the garbage on the conveying device;
the grabbing device is used for grabbing the target garbage;
the control unit comprises an image processing module for registering the obtained 2D image data and the 3D point cloud data to obtain RGB-D image data, and a deep learning training and calculating module for performing deep learning on the RGB-D image data to identify the material of the garbage to obtain target garbage and calculating the position information of the target garbage captured by the capturing device.
As a further improved technical scheme of the invention, the conveying device comprises a feeding end and a discharging end which are arranged oppositely, and the image acquisition device is arranged close to the feeding end.
As a further improved technical solution of the present invention, the image capturing device further includes a darkroom installed above the conveying device, and a light supplement device installed in the darkroom, the darkroom has a shooting opening opened toward the conveying device, and the line camera and the 3D camera are both installed in the darkroom.
As a further improved technical scheme of the invention, the conveying device comprises a conveying line and a driving assembly for driving the conveying line, wherein the driving assembly comprises a driving piece in communication connection with the control unit, a driving roller connected with the driving piece, and a driven roller linked with the driving roller; the vision and deep learning based garbage sorting system further comprises an encoder arranged on the driven roller; the encoder is in communication connection with the linear array camera, the 3D camera and the grabbing device.
As a further improved technical scheme of the invention, the number of the gripping devices is at least two, and the gripping devices are arranged side by side along the conveying direction of the conveying device.
As a further improved technical scheme of the invention, the control unit also comprises a grabbing control module; each group of the grabbing devices comprises a mounting frame, a manipulator and a plurality of garbage storage frames, wherein the manipulator is installed on the mounting frame and is in communication connection with the grabbing control module, and the garbage storage frames are used for placing sorted target garbage.
As a further improved technical scheme of the invention, the manipulator is a rectangular coordinate manipulator.
In order to achieve the above object, the present invention further provides a garbage sorting method of a garbage sorting system based on vision and deep learning, comprising the following steps:
acquiring 2D image data and 3D point cloud data of garbage to be sorted;
registering the 2D image data and the 3D point cloud data to obtain RGB-D image data;
loading a trained deep learning model to carry out deep learning on the RGB-D image data, and identifying the material of garbage to obtain target garbage;
calculating the position information of the target garbage grabbed by the grabbing device;
and (4) grabbing the target garbage and sorting the garbage.
As a further improved technical scheme of the invention, the deep learning model is a FasterR-CNN target detection algorithm model or an SSD target detection algorithm model.
As a further improved technical scheme of the invention, the steps of grabbing the target garbage and sorting the garbage specifically comprise the following steps:
along the conveying direction of the garbage, the garbage to be sorted sequentially passes through at least two grabbing devices to grab target garbage for garbage sorting.
The invention has the beneficial effects that: according to the garbage sorting system based on vision and deep learning, the material of garbage is recognized through deep learning of RGB-D image data to obtain target garbage, the position information of the target garbage captured by the capturing device is calculated, and the accuracy rate and the recognition rate of target garbage recognition are improved.
Drawings
Fig. 1 is a schematic structural diagram of a garbage sorting system based on visual and intelligent recognition technology in the present invention.
Fig. 2 is a block diagram schematically illustrating the structure of the visual and deep learning based garbage sorting system according to the present invention.
Fig. 3 is a flow chart of the garbage sorting method of the present invention.
Detailed Description
The present invention will be described in detail with reference to the embodiments shown in the drawings, and reference is made to fig. 1 to 3, which are preferred embodiments of the present invention. It should be noted that these embodiments are not intended to limit the present invention, and those skilled in the art should be able to make functional, methodical, or structural equivalents or substitutions according to these embodiments without departing from the scope of the present invention.
Referring to fig. 1 to 2, the present invention provides a garbage sorting system 100 based on vision and deep learning, which includes a conveying device 1 for conveying garbage to be sorted, an image acquiring device 2 for acquiring image data of the garbage to be sorted on the conveying device 1, a grabbing device 3 for grabbing target garbage, and a control unit 4, wherein the conveying device 1, the image acquiring device 2, and the grabbing device 3 are all connected to the control unit 4 in a communication manner.
Conveyor 1 includes conveyor belt 11, is used for the installation conveyor belt 11's carriage 12, is used for the drive conveyor belt 11 moves drive assembly (not reference numeral), drive assembly include with the driving piece that the control unit 4 communication is connected, with drive roll (not reference numeral) that the driving piece is connected, with the driven voller of drive roll linkage, control unit 4 control the driving piece drive the drive roll rotates, drives the driven voller rotates, makes conveyor belt 11 moves along direction of delivery to the transport of waiting to select separately rubbish is carried out.
In one embodiment, the driving member is a motor, but not limited thereto.
Specifically, conveyor belt 11 has relative feed end, the discharge end that sets up, treat the rubbish of sorting certainly the feed end gets into on the conveyor belt 11 the in-process of conveyor belt 11 operation drives rubbish to the discharge end removes.
Specifically, the image acquisition device 2 is arranged close to the feeding end so as to acquire image data of the garbage passing through in time in the process that the garbage to be sorted is conveyed from the feeding end to the discharging end.
The image acquisition device 2 comprises a linear array camera 21 for acquiring 2D image data of the garbage on the conveying device 1 and a 3D camera 22 for acquiring 3D point cloud data of the garbage on the conveying device 1, and the linear array camera 21 and the 3D camera 22 are both in communication connection with the control unit 4. When the garbage to be sorted passes through the image acquisition device 2, the linear array camera 21 continuously acquires and generates 2D image data of the passed garbage on line and feeds the data back to the control unit 4, and the 3D camera synchronously and continuously acquires and generates 3D point cloud data of the corresponding garbage on line and feeds the data back to the control unit 4.
It is understood that the 2D image data includes color information of an image, and the 3D point cloud data, i.e., three-dimensional information, includes information of a three-dimensional shape, a three-dimensional contour, a position, and the like.
Further, the garbage sorting system 100 based on vision and deep learning further comprises an encoder 5, and the encoder 5 is in communication connection with the line camera 21, the 3D camera 22 and the grabbing device 3.
The line camera 21 and the 3D camera 22 can respectively acquire image data of garbage at corresponding positions according to the electric signals fed back by the encoder 5, and feed back the image data to the control unit 4. That is, the 2D image data and the 3D point cloud data at the same position can be fed back to the control unit 4.
In one embodiment, the control unit 4 is a computer for controlling the operation of the garbage sorting system 100, and it is understood that the specific type of the computer is not specifically defined, and all computers capable of controlling the operation of the garbage sorting system 100 are within the scope of the present invention. Of course, this is not a limitation.
Further, the control unit 4 includes an image processing module (not shown) for aligning the acquired 2D image data with the 3D point cloud data to obtain RGB-D image data at a corresponding position, and a deep learning training and calculating module (not shown) for performing deep learning on the RGB-D image data to identify the material of the garbage to obtain target garbage and calculating the position information of the target garbage captured by the capturing device 3. The control unit 4 can feed back the obtained material of the target garbage and the position information of the grabbing device 3 grabbing the target garbage to the grabbing device 3, after the grabbing device 3 receives the material of the target garbage and the position information of the grabbing device 3 grabbing the target garbage, the running speed and the real-time position of the conveying belt 11 can be obtained according to the electric signal fed back by the encoder 5, and therefore the grabbing device 3 can dynamically grab the target garbage on the conveying belt 11, and online garbage sorting is achieved.
It is understood that the electric signal fed back by the encoder 5 means a signal including the running speed, the real-time position, and the like of the conveyor belt 11.
The target garbage is garbage to be sorted, such as wood boards, stones, plastics, paperboards and the like.
The RGB-D image data includes color information, three-dimensional position information, three-dimensional contour information, and the like of the garbage. In the invention, the material of the garbage is identified by deep learning the RGB-D image data to obtain the target garbage, and the accuracy and the identification rate of the target garbage identification are enhanced.
Specifically, a deep learning algorithm, such as the fast R-CNN target detection algorithm or the SSD target detection algorithm, is pre-stored in the deep learning training and calculating module to identify the material of the garbage in the acquired RGB-D image data, so as to obtain the target garbage.
In the following, the fast R-CNN target detection algorithm pre-stored in the deep learning training and calculating module is briefly described, but not limited thereto.
When the fast R-CNN target detection algorithm is prestored in the deep learning training and calculating module, the garbage treatment of the deep learning training and calculating module comprises the following steps:
s1: the deep learning training and calculating module acquires RGB-D image data input in a multi-dimensional array form, and the RGB-D image data is processed by a pre-trained CNN model to obtain a convolution characteristic diagram;
s2: processing the convolution characteristic diagram by RPN (region pro-position network), and identifying a region possibly containing target garbage;
s3: classifying the garbage in the boundary frame based on the R-CNN module, and identifying the material of the target garbage to obtain the target garbage;
s4: and calculating to obtain the position information of the target garbage captured by the capturing device.
After the deep learning training and calculation module identifies the material of the garbage to obtain the target garbage and calculates the position information of the grabbing device 3 for grabbing the target garbage, the material of the target garbage and the position information of the grabbing device 3 for grabbing the target garbage are fed back to the control unit 4, so that the grabbing device 3 is controlled to grab the target garbage subsequently.
Specifically, the encoder 5 is provided on the driven roller, and it is possible to prevent inaccuracy of the data collected due to slipping of the conveyor belt 11.
Further, the image capturing device 2 further includes a darkroom 23 installed above the conveying device 1, and a light supplement device (not shown) installed in the darkroom 23, wherein the darkroom 23 has a shooting opening opened toward the conveying device 1, and both the line camera 21 and the 3D camera 22 are installed in the darkroom 23, so as to prevent stray light from outside from affecting the shooting effects of the line camera 21 and the 3D camera 22.
Specifically, the dark room 23 is mounted on the conveying rack 12, and the dark room 23 has a photographing opening (not numbered) opened toward the conveying device 1 so that the line camera 21 and the 3D camera 22 acquire images of passing trash.
Further, grabbing device 3's quantity is at least two sets ofly, and at least two sets of grabbing device 3 follows conveyor belt 11's traffic direction interval and set up side by side to when the target rubbish is more, one grabbing device 3 comes not to all snatch the target rubbish, other grabbing device 3 can snatch remaining target rubbish, improves the efficiency and the rubbish sorting purity of rubbish sorting.
In a specific embodiment, the number of the gripping devices 3 is two, two gripping devices are defined as a first gripping device 31 and a second gripping device 32 along the conveying direction of the conveying belt 11, and during the conveying process of the conveying belt 11, the garbage to be sorted sequentially passes through the image acquisition device 2, the first gripping device 31 and the second gripping device 32, and finally the non-target garbage is discharged from the discharging end of the conveying device 1.
When the target garbage is large, and the garbage to be sorted passes through the first grabbing device 31, the first grabbing device 31 grabs part of the target garbage, and feeds back the material of the target garbage which is not grabbed and the position information of the target garbage grabbed by the second grabbing device 32 to the control unit 4, and the second grabbing device 32 dynamically grabs the rest target garbage according to the electric signal fed back by the encoder 5.
Further, the control unit 4 further includes a grabbing control module, each group of the grabbing devices 3 includes a mounting frame 33, a manipulator (not numbered) installed on the mounting frame 33 and in communication connection with the grabbing control module, and a plurality of garbage storage frames (not shown) for placing the sorted target garbage, each garbage storage frame is used for storing the target garbage of one type of material, and the manipulator grabs the target garbage and then recovers the target garbage into the corresponding garbage storage frame according to the material of the target garbage.
In a specific embodiment, the manipulator is a rectangular coordinate manipulator, has the characteristics of high speed, large load, large stroke and the like, and is suitable for sorting construction waste.
The rectangular coordinate manipulator comprises a rectangular coordinate module which is arranged on the mounting frame 33 and is provided with XYZR four-axis and a clamping jaw connected with the rectangular coordinate module, and the rectangular coordinate module can drive the clamping jaw to move in the direction of X, Y, Z, R four-axis so as to grab target garbage and place the target garbage into a corresponding garbage storage frame.
Specifically, the structure of the rectangular coordinate module can be the same as that of the existing rectangular coordinate module, and thus, the detailed description thereof is omitted.
Specifically, the mounting frame 33 is supported on the ground, so that the gripping devices 3 can be installed in a modular manner, that is, the mounting positions and the number of the gripping devices 3 can be selected according to specific requirements, and the installation is simple and convenient.
Further, referring to fig. 3, the present invention further provides a garbage sorting method of the garbage sorting system 100 based on vision and deep learning, which includes the following steps:
acquiring 2D image data and 3D point cloud data of garbage to be sorted;
registering the 2D image data and the 3D point cloud data to obtain RGB-D image data;
loading a trained deep learning model to carry out deep learning on the RGB-D image data, and identifying the material of garbage to obtain target garbage;
calculating the position information of the target garbage grabbed by the grabbing device;
and (4) grabbing the target garbage and sorting the garbage.
Specifically, the step of obtaining 2D image data and 3D point cloud data of the garbage to be sorted includes: the linear array camera 21 collects 2D image data at a certain position according to the electric signal fed back by the encoder 5, and the 3D camera 22 collects 3D point cloud data at a corresponding position according to the electric signal fed back by the encoder 5, and then feeds back the 2D image data and the 3D point cloud data at the same position to the control unit 4.
And loading the trained deep learning model to carry out deep learning on the RGB-D image data, and identifying the material of the garbage to obtain a target garbage deep learning model which is a FasterR-CNN target detection algorithm model or an SSD target detection algorithm model.
If the deep learning model is a FasterR-CNN target detection algorithm model, the step of loading the trained deep learning model to perform deep learning on the RGB-D image data and identifying the material of the garbage to obtain the target garbage specifically comprises the following steps:
s1: the deep learning training and calculating module acquires RGB-D image data input in a multi-dimensional array form, and the RGB-D image data is processed by a pre-trained CNN model to obtain a convolution characteristic diagram;
s2: processing the convolution characteristic diagram by RPN (region pro-position network), and identifying a region possibly containing target garbage;
s3: and classifying the garbage positioned in the boundary box based on the R-CNN module, and identifying the material of the target garbage to obtain the target garbage.
Specifically, the steps are as follows: snatch target rubbish, carry out rubbish and select separately and specifically include:
the control unit 4 feeds back the material of the target garbage and the position information of the target garbage grabbed by the grabbing device 3 to the grabbing device 3;
the grabbing device 3 obtains the running speed and the real-time position of the conveying belt 11 according to the electric signal fed back by the encoder 5, and dynamically grabs the target garbage on the conveying belt 11;
the grabbing device 3 puts the target garbage into the corresponding garbage storage frame according to the material of the grabbed target garbage.
Further, in an embodiment in which the number of the gripping devices 3 is multiple, and the multiple gripping devices 3 are arranged at intervals along the conveying direction of the conveying device 1, the step of gripping the target garbage includes the following steps:
the garbage to be sorted sequentially passes through at least two grabbing devices along the conveying direction to grab target garbage for sorting.
In a specific embodiment, the number of the gripping devices is two, two gripping devices are defined as a first gripping device 31 and a second gripping device 32 along the conveying direction of the conveying belt 11, and during the conveying process of the conveying belt 11, the garbage to be sorted sequentially passes through the first gripping device 31 and the second gripping device 32, and is finally discharged from the discharging end of the conveying device 1.
The method comprises the following steps of grabbing target garbage, and specifically carrying out garbage sorting, wherein the steps comprise:
the control unit 4 feeds back the material of the target garbage and the position information of the target garbage captured by the capturing device 3 to the first capturing device 31 and the second capturing device 32;
the first grabbing device 31 obtains the running speed and the real-time position of the conveying belt 11 according to the electric signal fed back by the encoder 5, dynamically grabs part of the target garbage on the conveying belt 11, puts the target garbage into a corresponding garbage storage frame according to the material of the grabbed target garbage, and feeds back the material of the target garbage which is not grabbed and the position information of the target garbage grabbed by the second grabbing device 32 to the second grabbing device 32 through the control unit 4;
the second grabbing device 31 obtains the running speed and the real-time position of the conveying belt 11 according to the electric signal fed back by the encoder 5, dynamically grabs the remaining target garbage on the conveying belt 11, and puts the target garbage into a corresponding garbage storage frame according to the material of the grabbed target garbage.
In summary, in the garbage sorting system 100 based on vision and deep learning in the present invention, the material of the garbage is recognized through deep learning of the RGB-D image data to obtain the target garbage, and the position information of the target garbage captured by the capturing device is calculated, so that the accuracy and the recognition rate of the target garbage recognition are enhanced.
It should be understood that although the present description refers to embodiments, not every embodiment contains only a single technical solution, and such description is for clarity only, and those skilled in the art should make the description as a whole, and the technical solutions in the embodiments can also be combined appropriately to form other embodiments understood by those skilled in the art.
The above-listed detailed description is only a specific description of a possible embodiment of the present invention, and they are not intended to limit the scope of the present invention, and equivalent embodiments or modifications made without departing from the technical spirit of the present invention should be included in the scope of the present invention.
Claims (10)
1. A vision and deep learning based garbage sorting system comprises a conveying device used for conveying garbage to be sorted; the method is characterized in that: the garbage sorting system based on vision and deep learning further comprises:
the image acquisition device comprises a linear array camera for acquiring 2D image data of the garbage on the conveying device and a 3D camera for acquiring 3D point cloud data of the garbage on the conveying device;
the grabbing device is used for grabbing the target garbage;
the control unit comprises an image processing module for registering the obtained 2D image data and the 3D point cloud data to obtain RGB-D image data, and a deep learning training and calculating module for performing deep learning on the RGB-D image data to identify the material of the garbage to obtain target garbage and calculating the position information of the target garbage captured by the capturing device.
2. The vision and deep learning based garbage sorting system of claim 1, wherein: the conveying device comprises a feeding end and a discharging end which are arranged oppositely, and the image acquisition device is arranged close to the feeding end.
3. The vision and deep learning based garbage sorting system of claim 2, wherein: the image acquisition device further comprises a darkroom arranged above the conveying device and a light supplementing device arranged in the darkroom, the darkroom is provided with a shooting opening which is opened towards the conveying device, and the linear array camera and the 3D camera are arranged in the darkroom.
4. The vision and deep learning based garbage sorting system of claim 1, wherein: the conveying device comprises a conveying line and a driving assembly used for driving the conveying line, wherein the driving assembly comprises a driving piece in communication connection with the control unit, a driving roller connected with the driving piece, and a driven roller linked with the driving roller; the vision and deep learning based garbage sorting system further comprises an encoder arranged on the driven roller; the encoder is in communication connection with the linear array camera, the 3D camera and the grabbing device.
5. The vision and deep learning based garbage sorting system of claim 1, wherein: the number of the gripping devices is at least two groups, and the gripping devices are arranged side by side along the conveying direction of the conveying device.
6. The vision and deep learning based garbage sorting system of claim 5, wherein: the control unit also comprises a grabbing control module; each group of the grabbing devices comprises a mounting frame, a manipulator and a plurality of garbage storage frames, wherein the manipulator is installed on the mounting frame and is in communication connection with the grabbing control module, and the garbage storage frames are used for placing sorted target garbage.
7. The vision and deep learning based garbage sorting system of claim 6, wherein: the manipulator is a rectangular coordinate manipulator.
8. A garbage sorting method of a garbage sorting system based on vision and deep learning is characterized in that: the method comprises the following steps:
acquiring 2D image data and 3D point cloud data of garbage to be sorted;
registering the 2D image data and the 3D point cloud data to obtain RGB-D image data;
loading a trained deep learning model to carry out deep learning on the RGB-D image data, and identifying the material of garbage to obtain target garbage;
calculating the position information of the target garbage grabbed by the grabbing device;
and (4) grabbing the target garbage and sorting the garbage.
9. The visual and deep learning based garbage sorting method of claim 8, further comprising: the deep learning model is a FasterR-CNN target detection algorithm model or an SSD target detection algorithm model.
10. The visual and deep learning based garbage sorting method of claim 8, further comprising: the method comprises the following steps of grabbing target garbage, and specifically carrying out garbage sorting, wherein the steps comprise:
along the conveying direction of the garbage, the garbage to be sorted sequentially passes through at least two grabbing devices to grab target garbage for garbage sorting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911196746.3A CN110743818A (en) | 2019-11-29 | 2019-11-29 | Garbage sorting system and garbage sorting method based on vision and deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911196746.3A CN110743818A (en) | 2019-11-29 | 2019-11-29 | Garbage sorting system and garbage sorting method based on vision and deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110743818A true CN110743818A (en) | 2020-02-04 |
Family
ID=69285140
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911196746.3A Pending CN110743818A (en) | 2019-11-29 | 2019-11-29 | Garbage sorting system and garbage sorting method based on vision and deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110743818A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111359915A (en) * | 2020-03-24 | 2020-07-03 | 广东弓叶科技有限公司 | Material sorting method and system based on machine vision |
| CN111659635A (en) * | 2020-06-16 | 2020-09-15 | 北京铮实环保工程有限公司 | Remaining garbage identification method and device based on visual technology and deep learning |
| CN111844101A (en) * | 2020-07-31 | 2020-10-30 | 中国科学技术大学 | Multi-finger dexterous hand sorting planning method |
| CN111921885A (en) * | 2020-08-05 | 2020-11-13 | 东南数字经济发展研究院江山分院 | Intelligent detection method and device for wood door plate size |
| CN113378868A (en) * | 2020-02-25 | 2021-09-10 | 顺丰科技有限公司 | Garbage sorting method and device, server and computer readable storage medium |
| CN113505633A (en) * | 2021-05-19 | 2021-10-15 | 海南师范大学 | Household garbage sorting treatment method and system based on block chain |
| KR20210130409A (en) * | 2020-04-22 | 2021-11-01 | 동명대학교산학협력단 | Classification System and method for atypical recycled goods using Deep learning |
| CN114055438A (en) * | 2022-01-17 | 2022-02-18 | 湖南视比特机器人有限公司 | Visual guide workpiece follow-up sorting system and method |
| KR20220077987A (en) * | 2020-12-02 | 2022-06-10 | 한국생산기술연구원 | Deep learning-based recyclables classification method |
| CN116921247A (en) * | 2023-09-15 | 2023-10-24 | 北京安麒智能科技有限公司 | Control method of intelligent garbage sorting system |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102218406A (en) * | 2011-01-04 | 2011-10-19 | 华南理工大学 | Intelligent detection device of defects of mobile phone outer shell based on machine vision |
| CN108447082A (en) * | 2018-03-15 | 2018-08-24 | 深圳市唯特视科技有限公司 | A kind of objective matching process based on combination learning Keypoint detector |
| CN109543601A (en) * | 2018-11-21 | 2019-03-29 | 电子科技大学 | A kind of unmanned vehicle object detection method based on multi-modal deep learning |
| CN109597087A (en) * | 2018-11-15 | 2019-04-09 | 天津大学 | A kind of 3D object detection method based on point cloud data |
| CN109834063A (en) * | 2019-04-11 | 2019-06-04 | 苏州嘉诺环境工程有限公司 | Garbage sorting system and refuse classification method |
| CN109977812A (en) * | 2019-03-12 | 2019-07-05 | 南京邮电大学 | A kind of Vehicular video object detection method based on deep learning |
| CN110378909A (en) * | 2019-06-24 | 2019-10-25 | 南京林业大学 | Single wooden dividing method towards laser point cloud based on Faster R-CNN |
| CN110516570A (en) * | 2019-08-15 | 2019-11-29 | 东莞弓叶互联科技有限公司 | A kind of garbage classification recognition methods of view-based access control model and device |
| CN111201451A (en) * | 2018-02-09 | 2020-05-26 | 宝马股份公司 | Method and device for detecting object in scene based on laser data and radar data of scene |
| CN211100232U (en) * | 2019-11-29 | 2020-07-28 | 苏州嘉诺环境工程有限公司 | Garbage sorting system based on vision and deep learning |
-
2019
- 2019-11-29 CN CN201911196746.3A patent/CN110743818A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102218406A (en) * | 2011-01-04 | 2011-10-19 | 华南理工大学 | Intelligent detection device of defects of mobile phone outer shell based on machine vision |
| CN111201451A (en) * | 2018-02-09 | 2020-05-26 | 宝马股份公司 | Method and device for detecting object in scene based on laser data and radar data of scene |
| CN108447082A (en) * | 2018-03-15 | 2018-08-24 | 深圳市唯特视科技有限公司 | A kind of objective matching process based on combination learning Keypoint detector |
| CN109597087A (en) * | 2018-11-15 | 2019-04-09 | 天津大学 | A kind of 3D object detection method based on point cloud data |
| CN109543601A (en) * | 2018-11-21 | 2019-03-29 | 电子科技大学 | A kind of unmanned vehicle object detection method based on multi-modal deep learning |
| CN109977812A (en) * | 2019-03-12 | 2019-07-05 | 南京邮电大学 | A kind of Vehicular video object detection method based on deep learning |
| CN109834063A (en) * | 2019-04-11 | 2019-06-04 | 苏州嘉诺环境工程有限公司 | Garbage sorting system and refuse classification method |
| CN110378909A (en) * | 2019-06-24 | 2019-10-25 | 南京林业大学 | Single wooden dividing method towards laser point cloud based on Faster R-CNN |
| CN110516570A (en) * | 2019-08-15 | 2019-11-29 | 东莞弓叶互联科技有限公司 | A kind of garbage classification recognition methods of view-based access control model and device |
| CN211100232U (en) * | 2019-11-29 | 2020-07-28 | 苏州嘉诺环境工程有限公司 | Garbage sorting system based on vision and deep learning |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113378868A (en) * | 2020-02-25 | 2021-09-10 | 顺丰科技有限公司 | Garbage sorting method and device, server and computer readable storage medium |
| CN111359915A (en) * | 2020-03-24 | 2020-07-03 | 广东弓叶科技有限公司 | Material sorting method and system based on machine vision |
| KR20210130409A (en) * | 2020-04-22 | 2021-11-01 | 동명대학교산학협력단 | Classification System and method for atypical recycled goods using Deep learning |
| KR102391501B1 (en) * | 2020-04-22 | 2022-04-27 | 동명대학교산학협력단 | Classification System and method for atypical recycled goods using Deep learning |
| CN111659635A (en) * | 2020-06-16 | 2020-09-15 | 北京铮实环保工程有限公司 | Remaining garbage identification method and device based on visual technology and deep learning |
| CN111844101A (en) * | 2020-07-31 | 2020-10-30 | 中国科学技术大学 | Multi-finger dexterous hand sorting planning method |
| CN111921885A (en) * | 2020-08-05 | 2020-11-13 | 东南数字经济发展研究院江山分院 | Intelligent detection method and device for wood door plate size |
| KR20220077987A (en) * | 2020-12-02 | 2022-06-10 | 한국생산기술연구원 | Deep learning-based recyclables classification method |
| KR102445891B1 (en) * | 2020-12-02 | 2022-09-23 | 한국생산기술연구원 | Deep learning-based recyclables classification method |
| CN113505633A (en) * | 2021-05-19 | 2021-10-15 | 海南师范大学 | Household garbage sorting treatment method and system based on block chain |
| CN114055438A (en) * | 2022-01-17 | 2022-02-18 | 湖南视比特机器人有限公司 | Visual guide workpiece follow-up sorting system and method |
| CN116921247A (en) * | 2023-09-15 | 2023-10-24 | 北京安麒智能科技有限公司 | Control method of intelligent garbage sorting system |
| CN116921247B (en) * | 2023-09-15 | 2023-12-12 | 北京安麒智能科技有限公司 | Control method of intelligent garbage sorting system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110743818A (en) | Garbage sorting system and garbage sorting method based on vision and deep learning | |
| CN106124520B (en) | A kind of device of full-automatic loading and unloading and vision-based detection | |
| CN107096720A (en) | Express mail bar code automatic recognition system and method based on image procossing | |
| CN109261539A (en) | A kind of garbage sorting system and method for view-based access control model identification and convolutional neural networks | |
| CN206854123U (en) | Express mail bar code automatic recognition system based on image procossing | |
| CN109625986A (en) | Robot palletizing system and its palletizing method | |
| CN111389763A (en) | Coal gangue sorting system and method based on image data processing | |
| CN105666485A (en) | Automatic identifying and positioning chess placing robot based on image processing | |
| JP2021030107A (en) | Article sorting apparatus, article sorting system, and article sorting method | |
| CN111266304A (en) | Coal briquette identification, detection and sorting system | |
| CN112916416A (en) | Building rubbish letter sorting system | |
| JP2021030219A (en) | Article sorting apparatus and article sorting method | |
| CN109834063A (en) | Garbage sorting system and refuse classification method | |
| CN111659635A (en) | Remaining garbage identification method and device based on visual technology and deep learning | |
| CN112827846B (en) | Automatic garbage classification device and method | |
| CN109433631A (en) | The integrated device that more sized bolt degree of injury detect and classify automatically | |
| CN102615053A (en) | Flexible printing machine added with quality inspection substandard-product rejecting device | |
| CN113289925A (en) | Intelligent sorting system and method for sundries in coal based on deep learning | |
| CN211100232U (en) | Garbage sorting system based on vision and deep learning | |
| CN209792032U (en) | Garbage sorting system based on vision and intelligent recognition technology | |
| CN208092786U (en) | A kind of the System of Sorting Components based on convolutional neural networks by depth | |
| CN113469264A (en) | Construction method of automatic garbage classification model, garbage sorting method and system | |
| CN111515133A (en) | Automatic sorting method, device and equipment for express special-shaped parcels and storage medium | |
| CN113996545A (en) | Quick sorting production line for main roots of panax notoginseng based on computer vision | |
| CN206701918U (en) | A kind of garbage sorting device of Multi-sensor Fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200204 |
|
| WD01 | Invention patent application deemed withdrawn after publication |