CN114548268A - Small sample garbage image classification method based on prototype network - Google Patents
Small sample garbage image classification method based on prototype network Download PDFInfo
- Publication number
- CN114548268A CN114548268A CN202210159436.XA CN202210159436A CN114548268A CN 114548268 A CN114548268 A CN 114548268A CN 202210159436 A CN202210159436 A CN 202210159436A CN 114548268 A CN114548268 A CN 114548268A
- Authority
- CN
- China
- Prior art keywords
- sample
- prototype
- network
- image
- garbage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02W—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO WASTEWATER TREATMENT OR WASTE MANAGEMENT
- Y02W90/00—Enabling technologies or technologies with a potential or indirect contribution to greenhouse gas [GHG] emissions mitigation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a small sample garbage image classification method based on a prototype network, which can finish automatic classification and identification of garbage images by needing a small amount of garbage image samples and can improve the automatic sorting efficiency in the garbage treatment process. The method comprises the following steps: the method comprises the steps of firstly, acquiring a plurality of garbage images by using a camera to construct a garbage classification data set, preprocessing the images by using a gamma correction method, then constructing a prototype network by using a four-layer convolutional neural network, then calculating a loss function between a query sample class probability output value and a real label value to train the prototype network, and finally fixing parameters in the prototype network to perform classification testing on the garbage images.
Description
Technical Field
The invention relates to the field of computer vision, in particular to a small sample junk image classification method based on a prototype network.
Background
With the rapid increase of economy, the total amount of municipal solid waste is also greatly increased year by year. The garbage is effectively and quickly treated in time, so that the method has important significance for creating beautiful living environment, saving earth resources and promoting the sustainable development of society. The automatic sorting of the garbage is a core link for realizing the garbage classification treatment, the classification and identification of the garbage images are the key of the automatic sorting link, and the identification speed directly restricts the efficiency of the whole garbage treatment production line.
With the advent of large-scale labeling data and the rapid development of high-performance graphics processors, deep convolutional neural networks have enjoyed great success in various computer vision fields such as image classification, target detection, and semantic segmentation. Researchers also apply the method to the field of automatic identification of garbage images, for example, Liu Guo dong and the like (Liu Guo dong, von Libra, Chenzi Jian, Liyi 2042928, Lu Tuo. A photoelectric intelligent garbage sorting method based on DMD and Yolov 5. application No. 202110758716.8) discloses a photoelectric intelligent garbage sorting method based on DMD and Yolov5, which inputs constructed garbage image data into a Yolov5 network for training to obtain a trained Yolov5 network as a garbage image identification model; exemplary Zhai-yi quai et al (Zhai-yi quai, Yu cuin, Keqirui, Zhouyou, Ganjin, Yinyang, Zeng military English, underwater vision garbage cleaning robot and its operation method, application No. 202010176992.9) discloses an underwater vision garbage cleaning robot and its operation method, the method uses an automatic encoder in a width learning network to extract garbage image characteristics, and uses a cost sensitive classification method to obtain weights and classify. The garbage sorting system and the garbage sorting method disclosed by Yuan Jing et al (Yuan Jing, Zhongxiang, Zhang Bian, Dun Yongcai. garbage sorting system and garbage sorting method. application No. 201910287785.) use a region convolution feature module in an Faster R-CNN model to extract convolution features of a target garbage image and classify and identify the convolution features.
However, the above method often depends on a large number of labeled images to complete the training of the model, and the amount of parameters in the model is large, so that a time lag is generated when the garbage images are classified and identified in real time, thereby affecting the sorting efficiency of the whole garbage processing generation line.
Disclosure of Invention
The purpose of the invention is as follows: in order to solve the defects in the prior art, the invention provides a small sample garbage image classification method based on a prototype network.
The technical scheme is as follows: the invention relates to a small sample junk image classification method based on a prototype network, which comprises the following main steps: step 1, image acquisition and pretreatment: the method comprises the steps of collecting a plurality of garbage images by using a camera to construct a garbage classification data set I, and dividing the data set I into a training set ItrainAnd test set Itest(ii) a Preprocessing each image in the image data set I by utilizing a gamma correction method to increase the brightness of the image so as to facilitate subsequent classification and identification;
step 2, constructing a prototype network: the network mainly comprises four modules, namely a backbone network, a prototype calculation module, a similarity calculation module and a Softmax layer;
step 3, inputting and outputting of the prototype network: constructing N-way-K-shot classification tasks on each training segment by adopting a segment type training mode, wherein each classification task utilizes a training set ItrainEstablishing a supporting sample data setAnd querying the sample data setWill be provided withAndthe image in the system is input into a prototype network, and the image sequentially passes through four modules of the network to obtain a class probability output value related to a query sample;
step 4, training a prototype network: calculating a loss function between the probability output value of the query sample class and the real label value, and optimizing parameters in the network by using a gradient descent method;
step 5, classification testing of the garbage images: fixing parameters in prototype networks using test data set ItestAnd establishing a support sample data set and a test sample data set, inputting the support sample data set and the test sample data set into a prototype network, and obtaining a category output value of the query sample sequentially through three modules of the network, thereby completing the classification test of the garbage image.
Preferably, the specific method for acquiring and preprocessing the image in the step 1 is as follows:
(1) the method comprises the steps of collecting a plurality of garbage images by using a camera to construct a garbage classification data set I, and randomly dividing the data set I into a training set ItrainAnd test set ItestManually labeling the images in the two data sets into four types, wherein the four types of labels are wet garbage, dry garbage, recoverable garbage and harmful garbage respectively;
(2) for each image of the image data set I, the nth image I is preprocessed by means of a gamma correction methodnThe formula for correction is:
Preferably, the specific method for constructing the prototype network in step 2 is as follows:
(1) the main network in the prototype network is composed of four convolution modules, each convolution module comprises a convolution layer with 64 3 multiplied by 3 filters, a batch normalization layer and a ReLU activation layer; in addition, the first two volume blocks are also added with a 2 multiplied by 2 maximum pool layer; the backbone network is denoted Fθ(. o), where θ represents a parameter in the backbone network;
(2) a prototype calculation module in the prototype network is represented as P (-), and the module carries out mean value calculation on the characteristics of each type of support sample;
(3) a similarity calculation module in the prototype network is represented as D (-), and calculates Euclidean distance between the query sample and each type of supporting sample prototype;
(4) the softmax layer in the prototype network is denoted as S (·), and the module converts the euclidean distance between the query sample and each type of supporting sample prototype into a probability output value.
Preferably, the specific input and output method of the prototype network in step 3 is as follows:
(1) in training set ItrainRandomly extracting N types of samples, and randomly extracting K samples from the N types of samples as a support sample data setThen randomly extracting Q samples from the rest samples to be used as a query sample set
(2) The kth corrected image sample in the support sample set is denoted asThe q-th image sample in the query sample set is represented asThe backbone networks via the prototype network are respectively represented asAndthe calculation formula of each type of prototype features when the support sample set passes through the prototype calculation module is as follows:
(3) when the support sample set and the query sample set pass through the similarity measurement module, a similarity measurement calculation formula for the qth image sample feature and the nth type of support prototype feature is as follows:
(4) the probability output value of the query sample set belonging to the nth category obtained by the softmax layer is represented as:
preferably, the specific training method of the prototype network in step 4 is as follows:
(1) the calculation formula of the loss function between the probability output value of the query sample class and the real label value is as follows:
wherein y isqnAn nth component representing a true label for the qth query sample;
(2) optimizing a parameter theta in the network by using a gradient descent method, wherein the formula for iterative calculation is as follows:
where λ is the iteration step parameter.
Preferably, the step 5 of the classification test of the spam images comprises the following specific steps:
(1) fixing a parameter theta in the prototype network;
(2) in test set ItestRandomly extracting N types of samples, and randomly extracting K samples from the N types of samples as a support sample data setThen in the remaining sampleRandomly extracting Q samples from the surface as a query sample set
(3) The kth corrected image sample in the support sample set is denoted asThe q-th image sample in the query sample set is represented asThe backbone networks via the prototype network are respectively represented asAndthe calculation formula of each type of prototype features when the support sample set passes through the prototype calculation module is as follows:
(4) when the support sample set and the query sample set pass through the similarity measurement module, a similarity measurement calculation formula for the qth image sample feature and the nth type of support prototype feature is as follows:
(4) the probability output value of the query sample set belonging to the nth category obtained by the softmax layer is represented as:
has the advantages that: the invention discloses a small sample garbage image classification method based on a prototype network, which can finish automatic classification and identification of garbage images by needing a small amount of garbage image samples and can improve the automatic sorting efficiency in the garbage processing process.
Drawings
Fig. 1 is a flowchart of a small sample spam image classification method based on a prototype network.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings; the embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
The detailed description of the present invention is illustrated with reference to fig. 1, which is a flowchart of a small sample spam image classification method based on a prototype network in fig. 1, and the detailed steps are as follows, as shown in fig. 1:
the method comprises the following steps: constructing a household garbage image data set and preprocessing images in the data set: the method comprises the steps of collecting images of garbage such as vegetable leaves, large bones, toothpaste boxes, waste batteries and the like by using a camera, manually marking the images into four categories of wet garbage, dry garbage, recoverable garbage and harmful garbage, constructing a garbage classification data set I, and dividing the garbage classification data set I into a training set ItrainAnd test set ItestThen, preprocessing by utilizing a gamma correction method;
step two: constructing a prototype network: the network is denoted F by the backbone network of parameter θθ(..), the prototype computation module is denoted as P (-), the similarity computation module D (-), and the softmax layer S (-);
step three: input and output of prototype network: in training set ItrainBuilding a set of supporting sample dataAnd querying the sample setThe kth support sample and the qth query sample are denoted asAndthe prototype of the nth class isSimilarity values between the qth query image sample feature and the nth supported prototype feature are then calculatedObtaining the probability output value of the q query image sample belonging to the nth category after passing through a Softmax layer
Step four: training a prototype network; calculating a loss function between a category probability output value and a real label value of a query sample, and optimizing a parameter theta in the network by using a gradient descent method;
step five, classification testing of the garbage images: in test set ItestOn-build support sample data setAnd querying the sample set, the backbone network pairs in the prototype networkAndthe kth support sample and the qth query sample are respectively represented asAndthe prototype of the nth class isSimilarity values between the qth query image sample feature and the nth supported prototype feature are then calculatedObtaining the probability output value of the q query image sample belonging to the nth category after passing through a Softmax layer
The above embodiments are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and any modifications made on the basis of the technical scheme according to the technical idea of the present invention fall within the protection scope of the present invention.
Claims (6)
1. The small sample junk image classification method based on the prototype network is characterized by comprising the following main steps:
step 1, image acquisition and pretreatment: the method comprises the steps of collecting a plurality of garbage images by using a camera to construct a garbage classification data set I, and dividing the data set I into a training set ItrainAnd test set Itest(ii) a Preprocessing each image in the image data set I by utilizing a gamma correction method to increase the brightness of the image so as to facilitate subsequent classification and identification;
step 2, constructing a prototype network: the network mainly comprises four modules, namely a backbone network, a prototype calculation module, a similarity calculation module and a Softmax layer;
step 3, inputting and outputting of the prototype network: constructing N-way-K-shot classification tasks on each training segment by adopting a segment type training mode, wherein each classification task utilizes a training set ItrainEstablishing a supporting sample data setAnd querying the sample data setWill be provided withAndthe image in the system is input into a prototype network, and the image sequentially passes through four modules of the network to obtain a class probability output value related to a query sample;
step 4, training a prototype network: calculating a loss function between the probability output value of the query sample class and the real label value, and optimizing parameters in the network by using a gradient descent method;
step 5, classification testing of the garbage images: fixing parameters in prototype networks using test data set ItestAnd establishing a support sample data set and a test sample data set, inputting the support sample data set and the test sample data set into a prototype network, and obtaining a category output value of the query sample sequentially through three modules of the network, thereby completing the classification test of the garbage image.
2. The small sample spam image classification method based on prototype network according to claim 1, wherein the specific method for image acquisition and preprocessing in step 1 is as follows:
(1) the method comprises the steps of collecting a plurality of garbage images by using a camera to construct a garbage classification data set I, and randomly dividing the data set I into a training set ItrainAnd test set ItestManually labeling the images in the two data sets into four types, wherein the four types of labels are wet garbage, dry garbage, recoverable garbage and harmful garbage respectively;
(2) for each image of the image data set I, the nth image I is preprocessed by means of a gamma correction methodnThe formula for correction is:
3. The small sample spam image classification method based on prototype network according to claim 1, wherein the specific construction method of the prototype network in the step 2 is as follows:
(1) the main network in the prototype network is composed of four convolution modules, each convolution module comprises a convolution layer with 64 3 multiplied by 3 filters, a batch normalization layer and a ReLU activation layer; in addition, the first two volume blocks are also added with a 2 multiplied by 2 maximum pool layer; the backbone network is denoted Fθ(. o), where θ represents a parameter in the backbone network;
(2) a prototype calculation module in the prototype network is represented as P (-), and the module carries out mean value calculation on the characteristics of each type of support sample;
(3) a similarity calculation module in the prototype network is represented as D (-), and calculates Euclidean distance between the query sample and each type of supporting sample prototype;
(4) the softmax layer in the prototype network is denoted as S (·), and the module converts the euclidean distance between the query sample and each type of supporting sample prototype into a probability output value.
4. The small sample spam image classification method based on prototype network according to claim 1, wherein the specific input and output methods of the prototype network in step 3 are as follows:
(1) in training set ItrainRandomly extracting N types of samples, and randomly extracting K samples from the N types of samples as a support sample data setThen randomly extracting Q samples from the rest samples to be used as a query sample set
(2) The kth corrected image sample in the support sample set is denoted asThe q-th image sample in the query sample set is represented asThe backbone networks through the prototype network are represented asAndthe calculation formula of each type of prototype features when the support sample set passes through the prototype calculation module is as follows:
(3) when the support sample set and the query sample set pass through the similarity measurement module, a similarity measurement calculation formula for the qth image sample feature and the nth type of support prototype feature is as follows:
(4) the probability output value of the query sample set belonging to the nth category obtained by the softmax layer is represented as:
5. the small sample spam image classification method based on prototype network according to claim 1, wherein the prototype network in step 4 is trained by the following specific method:
(1) the calculation formula of the loss function between the probability output value of the query sample class and the real label value is as follows:
wherein y isqnAn nth component representing a true label for the qth query sample;
(2) optimizing a parameter theta in the network by using a gradient descent method, wherein the formula for iterative calculation is as follows:
where λ is the iteration step parameter.
6. The small sample spam image classification method based on prototype network according to claim 1, wherein the classification test of spam images in step 5 is as follows:
(1) fixing a parameter theta in the prototype network;
(2) in test set ItestRandomly extracting N types of samples, and randomly extracting K samples from the N types of samples as a support sample data setThen randomly extracting Q samples from the rest samples to be used as a query sample set
(3) The kth corrected image sample in the support sample set is denoted asThe q-th image sample in the query sample set is represented asThe backbone networks via the prototype network are respectively represented asAndthe calculation formula of each type of prototype features when the support sample set passes through the prototype calculation module is as follows:
(4) when the support sample set and the query sample set pass through the similarity measurement module, a similarity measurement calculation formula for the qth image sample feature and the nth type of support prototype feature is as follows:
(4) the probability output value of the query sample set belonging to the nth category obtained by the softmax layer is represented as:
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210159436.XA CN114548268A (en) | 2022-02-16 | 2022-02-16 | Small sample garbage image classification method based on prototype network |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210159436.XA CN114548268A (en) | 2022-02-16 | 2022-02-16 | Small sample garbage image classification method based on prototype network |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114548268A true CN114548268A (en) | 2022-05-27 |
Family
ID=81677732
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210159436.XA Withdrawn CN114548268A (en) | 2022-02-16 | 2022-02-16 | Small sample garbage image classification method based on prototype network |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114548268A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115205521A (en) * | 2022-08-09 | 2022-10-18 | 湖南大学 | Kitchen waste detection method based on neural network |
CN115409124A (en) * | 2022-09-19 | 2022-11-29 | 小语智能信息科技(云南)有限公司 | Small sample sensitive information identification method based on fine-tuning prototype network |
-
2022
- 2022-02-16 CN CN202210159436.XA patent/CN114548268A/en not_active Withdrawn
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115205521A (en) * | 2022-08-09 | 2022-10-18 | 湖南大学 | Kitchen waste detection method based on neural network |
CN115205521B (en) * | 2022-08-09 | 2024-03-26 | 湖南大学 | Kitchen waste detection method based on neural network |
CN115409124A (en) * | 2022-09-19 | 2022-11-29 | 小语智能信息科技(云南)有限公司 | Small sample sensitive information identification method based on fine-tuning prototype network |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Su et al. | Deep learning-based solar-cell manufacturing defect detection with complementary attention network | |
CN108520780B (en) | Medical data processing and system based on transfer learning | |
CN111126386B (en) | Sequence domain adaptation method based on countermeasure learning in scene text recognition | |
CN114548268A (en) | Small sample garbage image classification method based on prototype network | |
CN114092832B (en) | High-resolution remote sensing image classification method based on parallel hybrid convolutional network | |
CN109919012A (en) | A kind of indicative microorganism image-recognizing method of sewage treatment based on convolutional neural networks | |
CN109800795A (en) | A kind of fruit and vegetable recognition method and system | |
CN111709477A (en) | Method and tool for garbage classification based on improved MobileNet network | |
CN111652835A (en) | Method for detecting insulator loss of power transmission line based on deep learning and clustering | |
CN115862073A (en) | Transformer substation harmful bird species target detection and identification method based on machine vision | |
CN110414587A (en) | Depth convolutional neural networks training method and system based on progressive learning | |
CN115131747A (en) | Knowledge distillation-based power transmission channel engineering vehicle target detection method and system | |
CN114329031A (en) | Fine-grained bird image retrieval method based on graph neural network and deep hash | |
CN114511532A (en) | Solar cell surface defect detection method based on feature-guided channel distillation | |
CN114399763A (en) | Single-sample and small-sample micro-body ancient biogenetic fossil image identification method and system | |
CN109471941A (en) | A kind of charge classification method for coping with class imbalance | |
CN111275025A (en) | Parking space detection method based on deep learning | |
CN116630700A (en) | Remote sensing image classification method based on introduction channel-space attention mechanism | |
CN113066049B (en) | MEMS sensor defect type identification method and system | |
CN115496948A (en) | Network supervision fine-grained image identification method and system based on deep learning | |
CN114548251A (en) | Small sample garbage image classification method based on regularization correlation network | |
CN114627333A (en) | Zinc flotation froth image classification algorithm and system for improving deep active learning | |
CN110033082B (en) | Method for identifying deep learning model in AI (Artificial intelligence) equipment | |
CN114139598A (en) | Fault diagnosis method and diagnosis framework based on deep cost sensitive convolution network | |
CN114295967A (en) | Analog circuit fault diagnosis method based on migration neural network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
WW01 | Invention patent application withdrawn after publication | ||
WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220527 |