CN110937280A - Audio-visual combination based intelligent garbage classification and recovery method and terminal - Google Patents
Audio-visual combination based intelligent garbage classification and recovery method and terminal Download PDFInfo
- Publication number
- CN110937280A CN110937280A CN201911347596.1A CN201911347596A CN110937280A CN 110937280 A CN110937280 A CN 110937280A CN 201911347596 A CN201911347596 A CN 201911347596A CN 110937280 A CN110937280 A CN 110937280A
- Authority
- CN
- China
- Prior art keywords
- garbage
- classification
- sub
- convolution
- outer shell
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65F—GATHERING OR REMOVAL OF DOMESTIC OR LIKE REFUSE
- B65F1/00—Refuse receptacles; Accessories therefor
- B65F1/14—Other constructional features; Accessories
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65F—GATHERING OR REMOVAL OF DOMESTIC OR LIKE REFUSE
- B65F1/00—Refuse receptacles; Accessories therefor
- B65F1/0033—Refuse receptacles; Accessories therefor specially adapted for segregated refuse collecting, e.g. receptacles with several compartments; Combination of receptacles
- B65F1/0053—Combination of several receptacles
- B65F1/006—Rigid receptacles stored in an enclosure or forming part of it
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65F—GATHERING OR REMOVAL OF DOMESTIC OR LIKE REFUSE
- B65F2210/00—Equipment of refuse receptacles
- B65F2210/138—Identification means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65F—GATHERING OR REMOVAL OF DOMESTIC OR LIKE REFUSE
- B65F2210/00—Equipment of refuse receptacles
- B65F2210/168—Sensing means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65F—GATHERING OR REMOVAL OF DOMESTIC OR LIKE REFUSE
- B65F2210/00—Equipment of refuse receptacles
- B65F2210/176—Sorting means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65F—GATHERING OR REMOVAL OF DOMESTIC OR LIKE REFUSE
- B65F2210/00—Equipment of refuse receptacles
- B65F2210/178—Steps
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02W—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO WASTEWATER TREATMENT OR WASTE MANAGEMENT
- Y02W30/00—Technologies for solid waste management
- Y02W30/10—Waste collection, transportation, transfer or storage, e.g. segregated refuse collecting, electric or hybrid propulsion
Abstract
The invention discloses an intelligent garbage classification and recovery method and a terminal based on audio-visual combination. Acquiring image number data of complete images of different materials of garbage, establishing a preprocessing convolutional neural network and a first sub-classification model and training; collecting sound signals of impact sounds of different materials of garbage, establishing a second sub-classification model and training; firstly, inputting the garbage into a pretreatment convolutional neural network to judge whether the garbage is monomer garbage or mixed garbage: and respectively inputting the image and the sound data into the first and second sub-classification models to be processed to obtain garbage type results of the two models, and performing weight synthesis to obtain a final result. The intelligent garbage classification terminal comprises a garbage can body, an automatic inductor, a classification mechanism, a garbage storage box and a control display. The invention can form the digital control of the whole garbage treatment production flow and improve the garbage treatment efficiency.
Description
Technical Field
The invention relates to a garbage classification processing method and a terminal, in particular to an intelligent garbage classification recycling method and a terminal based on audio-visual combination.
Background
With the explosive growth of worldwide garbage production over the years, garbage recycling has become a paramount concern in municipal solid waste management. The method relying on the self-conscious classification of residents when the residents throw garbage has a plurality of problems, such as high education cost, difficult supervision, difficult guarantee of classification accuracy and the like, and restricts the implementation of more elaborate garbage classification schemes. The existing intelligent garbage classification method is usually based on single signals such as vision and the like, has extremely high requirements on sample size, and is difficult to meet the requirements of real scenes. Aiming at the problems, the invention explores a novel intelligent garbage classification method, combines machine vision and hearing to help residents classify garbage quickly and accurately, and communicates data with a cloud end through an intelligent garbage classification terminal to realize data sharing and application from a garbage processing starting point.
Disclosure of Invention
In order to solve the problems in the background art, the invention aims to provide an intelligent garbage classification and recovery method and a terminal based on audio-visual combination, mainly solving a plurality of problems of a method for consciously classifying when people lose garbage, and achieving the purpose of helping people to classify the garbage quickly and accurately.
The technical scheme adopted by the invention is as follows:
an intelligent garbage classification recycling method based on audio-visual combination is shown in fig. 1:
1) acquiring image data of complete images of different materials of garbage, establishing a preprocessing convolutional neural network and a first sub-classification model, taking the image data as the input of the first sub-classification model, and taking the garbage type as a label of the first sub-classification model to train the first sub-classification model; the image data is used as the input of the preprocessing convolution neural network, and the garbage monomer garbage or the mixed garbage is used as the label of the preprocessing convolution neural network to train the preprocessing convolution neural network;
simultaneously collecting sound signals of impact sounds of different materials of garbage, preprocessing the sound signals to obtain sound data, establishing a second sub-classification model, taking the sound data as the input of the second sub-classification model, and taking the garbage type as a label of the second sub-classification model for training;
2) aiming at the garbage to be classified, after image data of the garbage to be classified is obtained, the image data is firstly input into a preprocessing convolution neural network to be judged to obtain monomer garbage or mixed garbage:
if the mixed garbage is mixed, the garbage is moved to a fixed position, and then the subsequent picking and sorting treatment is carried out;
if the garbage is monomer garbage, further judging the garbage type through an image or sound signal;
3) after the image and the sound data of the garbage to be classified are obtained, the image and the sound data are respectively input into a first sub-classification model and a second sub-classification model to be judged to obtain garbage type results of the two models, and the final garbage type result is obtained comprehensively according to different weights of the first sub-classification model and the second sub-classification model.
Therefore, the invention establishes a classification mode of audio-visual fusion, judges the garbage by using the results of images and sounds and improves the efficiency and the accuracy of identification processing.
The first sub-classification model and the second sub-classification model both adopt a convolutional neural network structure.
As shown in fig. 2, the first sub-classification model adopts a VGG-16 convolutional neural network structure, and mainly includes an input layer, two first convolutional pooling modules connected in sequence, three second convolutional pooling modules connected in sequence, two full-connection layers connected in sequence, and an output layer; each first convolution pooling module comprises two convolution layers and one pooling layer which are sequentially connected, the number of convolution kernels in the two convolution layers in the first convolution pooling module is 64, the kernel size is 3 multiplied by 3, the stride sequences are 1, the boundary padding is SAME, and the activation function is a ReLU function; two convolution layers in the second first convolution pooling module have the convolution kernel number of 128, the kernel size of 3 multiplied by 3, the stride strings of 1, the boundary padding of SAME, and the activation function of ReLU; each second convolution pooling module comprises three convolution layers and one pooling layer which are sequentially connected, the number of convolution kernels in the first convolution pooling module is 256, the kernel size is 3 multiplied by 3, the stride sequences is 1, the boundary padding is SAME, and the activation function is a ReLU function; the number of convolution kernels is 512, the kernel size is 3 multiplied by 3, the stride sequences is 1, the boundary padding is SAME, and the activation function is a ReLU function; five pooling layers in five convolutional pooling modules in sequence from an input layer to an output layer are maximum pooling layers, the number of convolutional kernels is 64, 128, 256, 512 and 512 respectively, the kernel size is 3 multiplied by 3, and the stride threads is 2; the two fully-connected layers both contain 4096 neurons, and the learned distributed features are mapped to a sample mark space; the output layer is classified by using a softmax function; and training the acquired image data of different material wastes by adopting a transfer learning method to obtain a first sub-classification model.
As shown in fig. 3, the second sub-classification model adopts a one-dimensional convolutional neural network, and mainly includes an input layer, five third convolutional pooling modules connected in sequence, two fully-connected layers, and an output layer, each third convolutional pooling module includes a convolutional layer and a pooling layer connected in sequence, five convolutional layers in the five third convolutional pooling modules in sequence from the input layer to the output layer, the number of convolutional kernels is 8, 16, 24, 32, and 40, the kernel size is 9 × 9, the stride strings is 1, the boundary padding is SAME, and the activation function is a ReLU function; five pooling layers in five third convolution pooling modules in sequence from the input layer to the output layer are maximum pooling layers, the number of convolution kernels is 8, 16, 24, 32 and 40 respectively, the kernel size is 9 multiplied by 9, and the stride crosses is 2; the fully connected layer contains 1024 neurons, and the learned distributed features are mapped to a sample mark space; the output layer is classified by using a softmax function; based on the network structure, sound signals of different materials of garbage are input for training to obtain a second sub-classification model.
The preprocessing convolutional neural network has the same structure as the first sub-classification model, and also adopts a VGG-16 convolutional neural network structure, and mainly comprises an input layer, 13 convolutional layers which are sequentially connected, 5 pooling layers which are sequentially connected, 3 full-connection layers which are sequentially connected and an output layer. The difference is that the input and the output are different, which finally results in different classification purposes of the preprocessing neural network and the first classification submodel.
Secondly, an intelligent garbage classification recycling terminal based on audio-visual combination: the intelligent garbage classification terminal comprises a garbage can body, a garbage delivery opening is formed in the upper portion of the garbage can body, an automatic sensor used for detecting objects is arranged above the garbage delivery opening, a classification mechanism is installed in the middle of the garbage can body, a plurality of garbage storage boxes are installed in the bottom of the garbage can body, a control display is installed in the upper portion of the garbage can body and is connected with the classification mechanism, the garbage storage boxes and the automatic sensor respectively.
The classification mechanism comprises an image sound acquisition device and a classification execution device, the image sound acquisition device comprises an outer shell, an upper port of the outer shell is opened, and the upper port of the outer shell is communicated with the garbage delivery port; the middle part of the outer shell is hinged with a rotary clapboard which divides the inner space of the outer shell into an upper space and a lower space which are respectively used as an image acquisition chamber and a sound acquisition chamber; a camera and an image acquisition room sensor are arranged on the inner wall of the outer shell in the image acquisition room; the inner wall of the outer shell in the sound collection chamber is provided with an upper sound collection chamber sensor, a lower sound collection chamber sensor, a microphone and an impact inclined plane; the lower half of the outer shell is provided with an opening, and the impact inclined plane is obliquely and fixedly arranged between the side wall of the outer shell at the bottom in the sound collection chamber and the bottom surface of the lower end without the opening; categorised final controlling element is including treating categorised room and lead screw slip table, and shell body lower extreme opening is through categorised passageway and treating categorised room intercommunication, treats categorised room and installs on the lead screw slip table, by the horizontal rectilinear movement of lead screw slip table drive lead screw slip table.
The rubbish bin include a plurality of dustbin, a plurality of dustbin arrange along lead screw slip table horizontal migration's straight line in proper order.
The middle part of the rotary clapboard is hinged between the inner walls at the two sides of the outer shell through a rotating shaft.
The camera is arranged on the inner wall of the outer shell at the top in the image acquisition room, and the image acquisition room sensor is arranged on the inner wall of the outer shell at the bottom in the image acquisition room and positioned beside the rotary partition plate; the sound collection room on the sensor and the microphone install in the shell body inner wall of the indoor middle part of sound collection, the sound collection room under the sensor install in the shell body inner wall of the indoor bottom of sound collection.
The bottom end of the chamber to be classified is provided with an opening cover which is hinged, a hinged shaft is connected with a driving motor, and the driving motor is connected to a control display.
The invention creatively utilizes the microphone to collect the sound signal of the garbage, uses the camera to collect the image of the garbage to be classified, and establishes a training preprocessing convolution neural network; respectively inputting image information and sound information into a classification model, establishing and utilizing a classification mechanism of audio-visual fusion to classify garbage, and throwing the garbage into a correct garbage storage box through a sorting mechanical structure.
The invention has the following beneficial effects:
(1) aiming at the problem that the current single classification method cannot meet the complex garbage classification scene, the method solves the problems that the method based on single image data is sensitive to the color, the shape, the deformation and the like of garbage, needs a great amount of training samples, needs to update a sample library in time and has high use difficulty in a real scene; the problem that the mixed garbage is difficult to accurately judge based on a single sound data method is solved; a more accurate garbage classification effect is achieved.
(2) The invention discloses an intelligent garbage classification terminal and an Internet of things platform design, wherein the intelligent garbage classification terminal based on sound and images is designed by combining advanced photovoltaic, edge calculation, Internet of things, automatic control and other technologies, the garbage type is automatically judged according to the input garbage, and the garbage type is distributed to sub-garbage boxes.
The invention is suitable for public places and family places, and meets the daily requirement of garbage classification. Meanwhile, wireless data intercommunication between the garbage classification terminal and the cloud platform and interaction between the garbage classification terminal and other equipment are achieved, real-time optimization of the internal space of the terminal is allowed, and reasonable planning of garbage recycling time and paths of the garbage truck is assisted. In addition, the method can predict the garbage yield by combining information such as climate, geography, environment, historical data and the like in the cloud platform, help subsequent processes to reasonably plan processing resources, achieve efficient garbage recycling, and lay a foundation for realizing a smart city.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a diagram of a first classification submodel and a preprocessed convolutional neural network design;
FIG. 3 is a convolutional neural network design diagram of a second classification submodel;
FIG. 4 is a schematic diagram of the whole intelligent garbage classification terminal;
FIG. 5 is a schematic diagram of a sorting mechanism of the intelligent garbage sorting terminal;
fig. 6 is a schematic diagram of an internet of things platform of the intelligent garbage classification terminal.
In the figure: 1. a dustbin body; 2. a trash storage bin; 3. a sorting mechanism; 4. a trash delivery opening; 5. an automatic sensor; 6. controlling the display; 31. an image and sound acquisition device; 32. a classification execution device; 310. an image acquisition room; 311. a sound collection chamber; 312. a sensor on the sound collection chamber; 313. a microphone; 314. a sound collection room lower sensor; 315. impacting the inclined plate; 316. turning over the partition board; 317. an image capture room sensor; 318. a camera; 320. a classification channel; 321. a room to be classified; 322. lead screw slip table.
Detailed Description
The present invention is further illustrated by the following figures and examples, which include, but are not limited to, the following examples.
As shown in fig. 3, intelligent waste classification terminal includes dustbin body 1, 1 upper portion outer wall of dustbin body is equipped with rubbish delivery opening 4, 1 upper portion outer wall of dustbin body above rubbish delivery opening 4 is equipped with automatic inductor 5 that is used for detecting the object, install sorting mechanism 3 in 1 middle part of dustbin body, install a plurality of rubbish bin 2 in 1 bottom of dustbin body, install control display 6 in 1 upper portion of dustbin body, control display 6 respectively with sorting mechanism 3, rubbish bin 2, automatic inductor 5 connects.
As shown in fig. 4, the sorting mechanism 3 includes an image and sound collecting device 31 for collecting image information and sound information of the garbage to be sorted, and a sorting executing device 32 for executing garbage sorting according to the type information of the garbage to be sorted. The image sound acquisition device 31 comprises an outer shell, an upper port of the outer shell is opened, and the upper port of the outer shell is communicated with the garbage delivery port 4; the middle part of the outer shell is hinged with a rotary clapboard 316, the middle part of the rotary clapboard 316 is hinged between the inner walls at two sides of the outer shell through a rotating shaft, and the rotary clapboard 316 divides the inner space of the outer shell into an upper space and a lower space which are respectively used as an image acquisition chamber 310 and a sound acquisition chamber 311; a camera 318 and an image acquisition chamber sensor 317 are arranged on the inner wall of the outer shell in the image acquisition chamber 310; the inner wall of the outer shell in the sound collection chamber 311 is provided with a sound collection chamber upper sensor 312, a sound collection chamber lower sensor 314, a microphone 313 and an impact inclined plane 315; the lower half of the outer shell is opened, and the impact inclined plane 315 is obliquely and fixedly arranged between the side wall of the outer shell at the bottom in the sound collection chamber 311 and the bottom surface of the outer shell with the lower end not opened; the classification executing device 32 comprises a chamber 321 to be classified and a screw rod sliding table 322, the lower end opening of the outer shell is connected and communicated with the chamber 321 to be classified through a vertical classification channel 320, the chamber 321 to be classified is installed on the screw rod sliding table 322, and the screw rod sliding table 322 drives the screw rod sliding table 322 to move horizontally and linearly.
The bottom end of the chamber to be classified 321 is provided with an opening cover which is hinged, the hinge shaft is connected with a driving motor, and the driving motor is connected to the control display 6 and is controlled by the control display 6 to open and close.
The garbage storage bin 2 includes a plurality of garbage cans which are sequentially arranged along the straight line of the horizontal movement of the screw rod sliding table 322.
The camera 318 is arranged on the inner wall of the outer shell at the top in the image acquisition room 310, and the image acquisition room sensor 317 is arranged on the inner wall of the outer shell at the bottom in the image acquisition room 310 and is positioned beside the rotary partition 316; the sound collection chamber upper sensor 312 and the microphone 313 are mounted on the inner wall of the outer housing at the middle part in the sound collection chamber 311, and the sound collection chamber lower sensor 314 is mounted on the inner wall of the outer housing at the bottom part in the sound collection chamber 311.
In a specific implementation, the sorting channel 320 is made of a flexible material, and the automatic sensor 5 is an infrared photoelectric sensor.
The garbage delivery and classification process of the invention comprises the following steps:
after the automatic inductor 5 detects there is the object, the automatic bounceing of the door of the refuse delivery opening 4, treat that the categorised rubbish carries out automatic classification through the sorting mechanism 3, the control display 6 will show the type information of treating the categorised rubbish to control the sorting mechanism 3 and throw the corresponding rubbish bin 2 with treating the categorised rubbish.
The garbage to be classified after collecting the image and sound information enters the chamber 321 to be classified through the classifiable channel 320, the display 6 is controlled to analyze the classification of the garbage, the screw rod sliding table 322 is driven to move the chamber 321 to be classified to the corresponding garbage classifying storage box 2, the lower end cover of the chamber 321 to be classified is opened, and the garbage to be classified is delivered.
When garbage objects to be classified reach the turnable clapboard 316 of the image acquisition chamber 310 through the garbage delivery opening 4, the image acquisition chamber sensor 317 is triggered, the camera 318 acquires images of the garbage to be classified, after the images are acquired, the turnable clapboard 316 is turned over, so that the garbage to be classified freely falls down and impacts the impact inclined plate 315, in the process, the sensor 312 on the sound acquisition chamber is triggered, the microphone 313 starts to record, and the recording is finished when the sensor 314 under the sound acquisition chamber is triggered. Finally, the collected images and sound of the garbage to be classified are sent to the control display 6 to be analyzed and processed to obtain the garbage category, and then the lead screw sliding table 322 is controlled to drive the garbage to be classified room 321 to move to the garbage classification storage box 2 corresponding to the garbage category to be put in, so that the garbage recovery efficiency is improved.
Control display 6 can be with the rubbish type that it produced, quantity, terminal remaining space, unable discernment object, information such as equipment trouble uploads to the high in the clouds in real time, establish the model data of control display 6 of every rubbish bin 2, combine weather simultaneously, geographical position, the environment of locating, factors such as future activity, realize the real time monitoring to every rubbish bin 2, problem prediction, and combine each terminal situation optimization rubbish recovery time of region and route, the control of terminal thing networking has been realized, promote rubbish recovery efficiency and management and control efficiency.
As shown in fig. 1, the embodiment and its implementation of the complete method according to the present invention are as follows:
the sound collection device built based on the terminal structure prepares various garbage materials, then collects and labels garbage sounds, preprocesses the data, and then trains a user network and a model.
1) Acquiring image data of complete images of different materials of garbage, establishing a preprocessing convolutional neural network and a first sub-classification model, taking the image data as the input of the first sub-classification model, and taking the garbage type as a label of the first sub-classification model to train the first sub-classification model; the image data is used as the input of the preprocessing convolution neural network, and the garbage monomer garbage or the mixed garbage is used as the label of the preprocessing convolution neural network to train the preprocessing convolution neural network;
simultaneously collecting sound signals of impact sounds of different materials of garbage, preprocessing the sound signals to obtain sound data, establishing a second sub-classification model, taking the sound data as the input of the second sub-classification model, and taking the garbage type as a label of the second sub-classification model for training;
2) aiming at the garbage to be classified, after image data of the garbage to be classified is obtained, the image data is firstly input into a preprocessing convolution neural network to be judged to obtain monomer garbage or mixed garbage:
if the mixed garbage is mixed, the garbage is moved to a fixed position, and then the subsequent picking and sorting treatment is carried out;
if the garbage is monomer garbage, further judging the garbage type through an image or sound signal;
3) after the image and the sound data of the garbage to be classified are obtained, the image and the sound data are respectively input into a first sub-classification model and a second sub-classification model to be judged to obtain garbage type results of the two models, and the final garbage type result is obtained comprehensively according to different weights of the first sub-classification model and the second sub-classification model.
As shown in fig. 6, concrete implementation is still through building thing networking platform, can gather rubbish data, and the comprehensive analysis realizes that urban garbage is more efficient to be handled, lays solid foundation for realizing the wisdom city:
firstly, twin data, a network and a model of the intelligent garbage classification terminal are established at the cloud end, and the internal space distribution of the terminal is optimized in real time according to the type quantity of the garbage to be thrown into the terminal, so that the space of the terminal is utilized to the maximum extent.
Meanwhile, the load condition of the terminal can guide the recovery route of the garbage truck, the running efficiency of the garbage truck is optimized, and garbage is cleaned in time. The government department can monitor the urban garbage condition in real time through the Internet of things platform, flexibly adjust the urban garbage disposal strategy and realize intelligent garbage disposal of the smart city.
In addition, peripheral data such as geography, environment, climate and activity information are combined, the garbage generation trend can be comprehensively analyzed, the total garbage discarding amount in a region can be predicted, and the garbage truck recycling frequency can be arranged in advance.
The above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiment, and any technical means that can achieve the object of the present invention by basically the same means is within the scope of the present invention.
Claims (10)
1. An intelligent garbage classification recycling method based on audio-visual combination is characterized in that:
1) acquiring image data of complete images of different materials of garbage, establishing a preprocessing convolutional neural network and a first sub-classification model, taking the image data as the input of the first sub-classification model, and taking the garbage type as a label of the first sub-classification model to train the first sub-classification model; the image data is used as the input of the preprocessing convolution neural network, and the garbage monomer garbage or the mixed garbage is used as the label of the preprocessing convolution neural network to train the preprocessing convolution neural network;
simultaneously collecting sound signals of impact sounds of different materials of garbage, preprocessing the sound signals to obtain sound data, establishing a second sub-classification model, taking the sound data as the input of the second sub-classification model, and taking the garbage type as a label of the second sub-classification model for training;
2) aiming at the garbage to be classified, after image data of the garbage to be classified is obtained, the image data is firstly input into a preprocessing convolution neural network to be judged to obtain monomer garbage or mixed garbage:
if the mixed garbage is mixed, the garbage is moved to a fixed position, and then the subsequent picking and sorting treatment is carried out;
if the garbage is monomer garbage, further judging the garbage type through an image or sound signal;
3) after the image and the sound data of the garbage to be classified are obtained, the image and the sound data are respectively input into a first sub-classification model and a second sub-classification model to be judged to obtain garbage type results of the two models, and the final garbage type result is obtained comprehensively according to different weights of the first sub-classification model and the second sub-classification model.
2. The intelligent garbage classification and recovery processing method based on audio-visual combination according to claim 1, characterized in that: the first sub-classification model and the second sub-classification model both adopt a convolutional neural network structure.
3. The intelligent garbage classification and recovery processing method based on audio-visual combination according to claim 2, characterized in that: the first sub-classification model adopts a VGG-16 convolutional neural network structure and mainly comprises an input layer, two first convolutional pooling modules which are connected in sequence, three second convolutional pooling modules which are connected in sequence, two full-connection layers and an output layer which are connected in sequence; each first convolution pooling module comprises two convolution layers and one pooling layer which are sequentially connected, the number of convolution kernels in the two convolution layers in the first convolution pooling module is 64, the kernel size is 3 multiplied by 3, the stride sequences are 1, the boundary padding is SAME, and the activation function is a ReLU function; two convolution layers in the second first convolution pooling module have the convolution kernel number of 128, the kernel size of 3 multiplied by 3, the stride strings of 1, the boundary padding of SAME, and the activation function of ReLU; each second convolution pooling module comprises three convolution layers and one pooling layer which are sequentially connected, the number of convolution kernels in the first convolution pooling module is 256, the kernel size is 3 multiplied by 3, the stride sequences is 1, the boundary padding is SAME, and the activation function is a ReLU function; the number of convolution kernels is 512, the kernel size is 3 multiplied by 3, the stride sequences is 1, the boundary padding is SAME, and the activation function is a ReLU function; five pooling layers in five convolutional pooling modules in sequence from an input layer to an output layer are maximum pooling layers, the number of convolutional kernels is 64, 128, 256, 512 and 512 respectively, the kernel size is 3 multiplied by 3, and the stride threads is 2; the two full-connection layers respectively contain 4096 neurons, and the output layer is classified by using a softmax function;
the second sub-classification model adopts a one-dimensional convolutional neural network and mainly comprises an input layer, five third convolutional pooling modules, two full-connection layers and an output layer, wherein the five third convolutional pooling modules are sequentially connected with one another, each third convolutional pooling module comprises a convolutional layer and a pooling layer, the number of convolutional cores in the five third convolutional pooling modules is 8, 16, 24, 32 and 40, the core size is 9 x 9, the stride strings is 1, the boundary padding is SAME, and the activation function is a ReLU function; five pooling layers in five third convolution pooling modules in sequence from the input layer to the output layer are maximum pooling layers, the number of convolution kernels is 8, 16, 24, 32 and 40 respectively, the kernel size is 9 multiplied by 9, and the stride crosses is 2; the fully-connected layer contains 1024 neurons, and the output layer is classified using the softmax function.
4. The intelligent garbage classification and recovery processing method based on audio-visual combination according to claim 3, characterized in that: the preprocessing convolution neural network has the same structure as the first sub-classification model.
5. An intelligent garbage classification recycling terminal based on audio-visual combination, which is applied to the method of claim 1, and is characterized in that: intelligence waste classification terminal include dustbin body (1), dustbin body (1) upper portion is equipped with rubbish delivery opening (4), rubbish delivery opening (4) top is equipped with automatic induction ware (5) that are used for detecting the object, install classification mechanism (3) in dustbin body (1) middle part, install a plurality of rubbish bin (2) in dustbin body (1) bottom, install control display (6) in dustbin body (1) upper portion, control display (6) respectively with classification mechanism (3), rubbish bin (2), automatic induction ware (5) are connected.
6. The intelligent garbage classification and recycling terminal based on audio-visual combination according to claim 5, characterized in that: the classification mechanism (3) comprises an image and sound acquisition device (31) and a classification execution device (32), the image and sound acquisition device (31) comprises an outer shell, an upper port of the outer shell is opened, and the upper port of the outer shell is communicated with the garbage delivery port (4); the middle part of the outer shell is hinged with a rotary clapboard (316), the inner space of the outer shell is divided into an upper space and a lower space by the rotary clapboard (316), and the upper space and the lower space are respectively used as an image acquisition chamber (310) and a sound acquisition chamber (311); a camera (318) and an image acquisition chamber sensor (317) are arranged on the inner wall of the outer shell in the image acquisition chamber (310); the inner wall of the outer shell in the sound collection chamber (311) is provided with a sound collection chamber upper sensor (312), a sound collection chamber lower sensor (314), a microphone (313) and an impact inclined plane (315); the lower end of the outer shell is half opened, and the impact inclined plane (315) is obliquely and fixedly arranged between the side wall of the outer shell at the bottom in the sound collection chamber (311) and the bottom surface of the outer shell with the lower end not opened; the classification execution device (32) comprises a chamber (321) to be classified and a screw rod sliding table (322), an opening at the lower end of the outer shell is communicated with the chamber (321) to be classified through a classification channel (320), the chamber (321) to be classified is installed on the screw rod sliding table (322), and the screw rod sliding table (322) is driven to horizontally and linearly move by the screw rod sliding table (322).
7. The intelligent garbage classification and recycling terminal based on audio-visual combination according to claim 6, characterized in that: the garbage storage box (2) comprises a plurality of garbage boxes, and the garbage boxes are sequentially arranged along the straight line of the horizontal movement of the screw rod sliding table (322).
8. The intelligent garbage classification and recycling terminal based on audio-visual combination according to claim 6, characterized in that: the middle part of the rotary clapboard (316) is hinged between the inner walls at the two sides of the outer shell through a rotating shaft.
9. The intelligent garbage classification and recycling terminal based on audio-visual combination according to claim 6, characterized in that: the camera (318) is arranged on the inner wall of the outer shell at the top in the image acquisition room (310), and the image acquisition room sensor (317) is arranged on the inner wall of the outer shell at the bottom in the image acquisition room (310) and is positioned beside the rotary partition plate (316); the sound collection room upper sensor (312) and the microphone (313) are arranged on the inner wall of the outer shell at the middle part in the sound collection room (311), and the sound collection room lower sensor (314) is arranged on the inner wall of the outer shell at the bottom in the sound collection room (311).
10. The intelligent garbage classification and recycling terminal based on audio-visual combination according to claim 6, characterized in that: the bottom end of the chamber (321) to be classified is provided with an opening cover which is hinged, a hinged shaft is connected with a driving motor, and the driving motor is connected with a control display (6).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911347596.1A CN110937280B (en) | 2019-12-24 | 2019-12-24 | Audio-visual combination based intelligent garbage classification and recovery method and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911347596.1A CN110937280B (en) | 2019-12-24 | 2019-12-24 | Audio-visual combination based intelligent garbage classification and recovery method and terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110937280A true CN110937280A (en) | 2020-03-31 |
| CN110937280B CN110937280B (en) | 2021-11-16 |
Family
ID=69912663
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911347596.1A Active CN110937280B (en) | 2019-12-24 | 2019-12-24 | Audio-visual combination based intelligent garbage classification and recovery method and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110937280B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111470206A (en) * | 2020-04-27 | 2020-07-31 | 中南大学湘雅三医院 | Voice recognition medical garbage treatment device and method |
| CN111717560A (en) * | 2020-06-14 | 2020-09-29 | 武汉理工大学 | Intelligent classification garbage bin based on computer vision technique |
| CN112278636A (en) * | 2020-10-15 | 2021-01-29 | 张良虎 | Garbage classification recycling method, device and system and storage medium |
| WO2021129236A1 (en) * | 2019-12-24 | 2021-07-01 | 浙江大学 | Acoustic garbage classification method using one-dimensional convolutional neural network |
| WO2021220987A1 (en) * | 2020-04-30 | 2021-11-04 | Ｊｆｅスチール株式会社 | Scrap determination system and scrap determination method |
| CN113877941A (en) * | 2021-10-29 | 2022-01-04 | 张秋艳 | Low-carbon treatment process for automatic sorting and resource circulation of household garbage |
| CN114512153A (en) * | 2022-01-17 | 2022-05-17 | 浙江大学 | Audio-visual touch data acquisition terminal for garbage |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107590830A (en) * | 2017-09-21 | 2018-01-16 | 北京小米移动软件有限公司 | Method for determining height, device and storage medium based on image |
| CN109292300A (en) * | 2018-11-02 | 2019-02-01 | 许诺石 | A kind of recyclable device of actively classifying |
| CN109969655A (en) * | 2019-04-16 | 2019-07-05 | 南通理工学院 | A kind of Intelligent refuse classification processing system |
| CN110210635A (en) * | 2019-06-05 | 2019-09-06 | 周皓冉 | A kind of intelligent classification recovery system that can identify waste |
| CN110428019A (en) * | 2019-08-09 | 2019-11-08 | 绵阳德川鸿丰环保科技有限公司 | Intelligent garbage classification method and modularization intelligent garbage classification processing system |
-
2019
- 2019-12-24 CN CN201911347596.1A patent/CN110937280B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107590830A (en) * | 2017-09-21 | 2018-01-16 | 北京小米移动软件有限公司 | Method for determining height, device and storage medium based on image |
| CN109292300A (en) * | 2018-11-02 | 2019-02-01 | 许诺石 | A kind of recyclable device of actively classifying |
| CN109969655A (en) * | 2019-04-16 | 2019-07-05 | 南通理工学院 | A kind of Intelligent refuse classification processing system |
| CN110210635A (en) * | 2019-06-05 | 2019-09-06 | 周皓冉 | A kind of intelligent classification recovery system that can identify waste |
| CN110428019A (en) * | 2019-08-09 | 2019-11-08 | 绵阳德川鸿丰环保科技有限公司 | Intelligent garbage classification method and modularization intelligent garbage classification processing system |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021129236A1 (en) * | 2019-12-24 | 2021-07-01 | 浙江大学 | Acoustic garbage classification method using one-dimensional convolutional neural network |
| US11835489B2 (en) | 2019-12-24 | 2023-12-05 | Zhejiang University | Acoustic garbage classification method using one-dimensional convolutional neural network (1D-CNN) |
| CN111470206A (en) * | 2020-04-27 | 2020-07-31 | 中南大学湘雅三医院 | Voice recognition medical garbage treatment device and method |
| WO2021220987A1 (en) * | 2020-04-30 | 2021-11-04 | Ｊｆｅスチール株式会社 | Scrap determination system and scrap determination method |
| CN111717560A (en) * | 2020-06-14 | 2020-09-29 | 武汉理工大学 | Intelligent classification garbage bin based on computer vision technique |
| CN112278636A (en) * | 2020-10-15 | 2021-01-29 | 张良虎 | Garbage classification recycling method, device and system and storage medium |
| CN113877941A (en) * | 2021-10-29 | 2022-01-04 | 张秋艳 | Low-carbon treatment process for automatic sorting and resource circulation of household garbage |
| CN113877941B (en) * | 2021-10-29 | 2022-11-01 | 张秋艳 | Low-carbon treatment process for automatic sorting and resource circulation of household garbage |
| CN114512153A (en) * | 2022-01-17 | 2022-05-17 | 浙江大学 | Audio-visual touch data acquisition terminal for garbage |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110937280B (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110937280B (en) | Audio-visual combination based intelligent garbage classification and recovery method and terminal | |
| WO2020206862A1 (en) | Automatic sorting system | |
| CN109606991B (en) | Intelligent garbage can and garbage classification method based on deep learning | |
| CN110884791A (en) | Vision garbage classification system and classification method based on TensorFlow | |
| CN111498331B (en) | Garbage visual recognition method and device based on deep learning and classified recovery device | |
| CN110294236A (en) | A kind of garbage classification monitoring apparatus, method and server system | |
| CN109305490B (en) | Intelligent classification dustbin based on image recognition technology | |
| CN110589282A (en) | Intelligent garbage classification method based on machine learning and automatic garbage sorting device | |
| CN110654739A (en) | Machine vision-based automatic recyclable garbage classification recycling device and method | |
| CN211979700U (en) | Sharing cloud intelligence garbage bin that can independently walk | |
| CN110355188B (en) | Smart garbage classification in-situ reduction recycling system based on multiple land utilization | |
| CN109573403A (en) | Novel and multifunctional intelligent dustbin and application method | |
| CN111559588A (en) | Intelligent garbage can for classified garbage throwing and classified garbage throwing method | |
| CN111661508B (en) | Intelligent garbage classification recycling device based on interaction of Internet and Internet of things | |
| CN110861853B (en) | Intelligent garbage classification method combining vision and touch | |
| CN109809060A (en) | A kind of automatic sorting garbage bin and its management system | |
| CN111186656A (en) | Target garbage classification method and intelligent garbage can | |
| CN110466911A (en) | Automatic sorting garbage bin and classification method | |
| CN112591333A (en) | Automatic garbage classification device and method based on artificial intelligence | |
| CN113859803A (en) | Intelligent identification trash can and intelligent identification method thereof | |
| CN111137602B (en) | Garbage classification method and garbage classification system based on artificial intelligence | |
| CN113320861A (en) | Traceable contactless garbage classification control system carrying mobile vehicle | |
| CN211418362U (en) | Artificial intelligence classification garbage can | |
| CN216470019U (en) | Domestic waste intelligent classification recovery system based on internet | |
| CN112620165B (en) | Garbage classification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |