CN115147703A - GinTrans network-based garbage segmentation method and system - Google Patents
GinTrans network-based garbage segmentation method and system Download PDFInfo
- Publication number
- CN115147703A CN115147703A CN202210901322.8A CN202210901322A CN115147703A CN 115147703 A CN115147703 A CN 115147703A CN 202210901322 A CN202210901322 A CN 202210901322A CN 115147703 A CN115147703 A CN 115147703A
- Authority
- CN
- China
- Prior art keywords
- feature
- module
- output
- frequency
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000012545 processing Methods 0.000 claims abstract description 46
- 230000004927 fusion Effects 0.000 claims abstract description 28
- 238000005520 cutting process Methods 0.000 claims abstract description 19
- 238000007634 remodeling Methods 0.000 claims abstract description 13
- 238000003709 image segmentation Methods 0.000 claims abstract description 9
- 238000000605 extraction Methods 0.000 claims description 26
- 238000005070 sampling Methods 0.000 claims description 24
- 238000010586 diagram Methods 0.000 claims description 21
- 238000011176 pooling Methods 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 16
- 238000010606 normalization Methods 0.000 claims description 15
- 230000002776 aggregation Effects 0.000 claims description 12
- 238000004220 aggregation Methods 0.000 claims description 12
- 238000000354 decomposition reaction Methods 0.000 claims description 9
- 230000007246 mechanism Effects 0.000 claims description 9
- 230000006798 recombination Effects 0.000 claims description 9
- 238000005215 recombination Methods 0.000 claims description 9
- 238000011084 recovery Methods 0.000 claims description 9
- 230000009191 jumping Effects 0.000 claims description 8
- 230000009471 action Effects 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 239000003550 marker Substances 0.000 claims description 6
- 238000004064 recycling Methods 0.000 claims description 6
- 239000000463 material Substances 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 abstract description 5
- 238000005065 mining Methods 0.000 abstract description 3
- 230000004044 response Effects 0.000 abstract description 3
- 238000006116 polymerization reaction Methods 0.000 abstract 1
- 230000006872 improvement Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000003203 everyday effect Effects 0.000 description 2
- 238000003915 air pollution Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The invention discloses a garbage segmentation method based on a GinTrans network, which comprises the following steps: (1) setting an image segmentation system; (2) image acquisition and input; (3) extracting a feature map; (4) cutting, marking and recombining; (5) remodeling fusion; (6) skip polymerization; and (7) dividing and outputting. The invention also discloses an image segmentation system. According to the invention, the feature graph required by feature mining is generated through linear transformation, so that network parameters are reduced, the complexity is reduced, the segmentation processing efficiency is ensured, and the response speed is increased; the characteristic of Frequency division processing of high Frequency and low Frequency is realized by utilizing a Bi-Frequency Transformer (BiFTrans block) module, the capturing of high Frequency details and the consideration of low Frequency global information are realized, the speed and the stability of image processing are ensured, the precision and the accuracy of garbage target segmentation are improved, and the classification efficiency is improved.
Description
Technical Field
The invention relates to the technical field of visual processing, in particular to a method and a system for garbage segmentation based on a GinTrans network.
Background
Along with the rapid development of economy, the living standard of people is improved, the types of generated garbage are more and more complex, and the unified landfill method cannot meet the concept of protecting the natural and green development. The classification treatment of the garbage can realize the recycling of resources, reduce the harm of soil and prevent air pollution. To realize an environment-friendly society and realize the recycling of resources, garbage classification is an important step. The garbage classification is started from each small family and each person, so that a large amount of manpower and material resources can be saved in the subsequent work. The recycling of the recyclable garbage, the correct putting of non-recyclable matters and the garbage treatment are realized. At present, in 46 key cities which are developed by China and are tested in advance in the current garbage classification, the coverage rate of a household garbage classification cell reaches 86.6%, the average recycling rate of household garbage is 30.4%, and the kitchen garbage treatment capacity is improved from 3.47 ten thousand tons every day in 2019 to 6.28 ten thousand tons every day in 2020 years.
However, although the performance of domestic garbage classification and treatment is good in recent years, garbage classification in partial cities and communities is still a difficult problem, manual sorting is still needed in most garbage classification centers, the efficiency is low, and the human body is injured due to sharp garbage scratches. Through artificial intelligence technique, utilize computer vision and image processing to cut apart rubbish, not only can promote rubbish letter sorting efficiency greatly, but also accurate classification will mix recoverable rubbish screening out in other rubbish, realize the high efficiency and the cyclic utilization of resource, but the following problem that still exists of the rubbish classification system based on vision that nevertheless current rubbish adopted: 1. the existing garbage segmentation method based on traditional image processing has low segmentation precision; 2. the existing segmentation method based on the traditional deep learning network has the defects of more model parameters, high calculation complexity and influence on the subsequent image processing speed and efficiency; 3. the garbage targets are various in types, serious in deformation and pollution, the boundaries between the targets are not cleaned, and the existing method cannot capture detail and outline information well, so that the segmentation difficulty is high, the miscut segmentation rate is high, and the subsequent classification precision is influenced.
Disclosure of Invention
The invention provides a GinTrans network-based garbage segmentation method aiming at the defects of the prior art.
The invention also discloses an image segmentation system.
The technical scheme adopted by the invention for realizing the purpose is as follows:
a garbage segmentation method based on a GinTrans network comprises the following steps:
(1) The image segmentation system is set up as follows: the image segmentation system comprises an image acquisition module, an extraction module, a cutting identifier recombination module, a remodeling fusion output module, a jumping connection aggregation module and a segmentation output module, wherein the extraction module is provided with a GhostNet network, the remodeling fusion output module is provided with a Bi-Frequency transform (BiFTrans) module, the jumping connection aggregation module is provided with an upper sampling recovery module and a GhostNet down-sampling feature extraction module, and the segmentation output module is provided with a segmentation head module;
(2) Image acquisition input: the image acquisition module is connected with an image acquisition camera, the image acquisition camera acquires images of the garbage on the intelligent garbage recycling and sorting line and transmits the images to the image acquisition module, and the images acquired by the image acquisition module are used as image input data and are transmitted to the extraction module for processing;
(3) Extracting a characteristic diagram: the extraction module processes the image input data, and the extraction module adopts a GhostNet network to extract bottom layer features of the image input data, extracts an output feature map and carries out the next step of processing;
(4) Cutting mark recombination: the cutting identification recombination module is used for cutting the extracted output characteristic diagram, gridding the output characteristic diagram through cutting operation, generating marking characteristics through linear mapping after gridding is finished, forming marking characteristic sequences by the marking characteristics, and taking the marking characteristic sequences as the input of a Bi-Frequency transform (BiFTrans block) module;
(5) Remodeling and fusion: the remodeling fusion output module remodels the input marked feature sequence, and utilizes a dual-Frequency mixer arranged in a Bi-Frequency transform (BiFTrans block) module to perform fusion action on the features from different high and low frequencies, and the fusion action obtains fusion feature output;
(6) Jumping aggregation: recovering the fused feature output into an input image with the same size as the image input data through an up-sampling recovery module, extracting the lower material features of the corresponding same-layer resolution feature map through a GhostNet down-sampling feature extraction module, and jumping and linking the input image obtained by the up-sampling recovery module and the same-layer resolution feature map obtained by the GhostNet down-sampling feature extraction module so as to aggregate feature images presented in different resolution levels;
(7) And (3) segmentation output: and the segmentation head module of the segmentation output module generates and outputs the feature image to obtain a target image.
The improvement is further that the GhostNet network is provided with a Ghost module, and the design of the Ghost module is as follows:
(3.1) carrying out convolution on the input data of the to-be-identified garbage image in the step (2) through a GhostNet network to generate original feature maps f with different resolutions i ′；
(3.2) the extraction module contains linear operation phi, and the linear operation phi is utilized to generate a Ghost characteristic diagram as an output characteristic diagram, wherein the formula is f ij ＝Φ i，j (f i ') wherein, f i ' is the ith original characteristic map, phi, obtained by convolution operation of the garbage image to be identified i，j Performing j linear operation on the ith original feature map to obtain a j Ghost feature map f ij 。
In a further improvement, the step (4) further comprises the following steps:
(4.1) performing gridding cutting on each Ghost feature map output in the step (3), wherein the size of each feature map is H multiplied by W, the H and the W are respectively the length and the width of the feature map, the size of each grid sub-feature map is s multiplied by s, the s is the side length of a grid, and the s can be divided by the H and the W into grid sub-feature maps;
(4.2) carrying out linear mapping on the gridded sub-feature graphs, wherein each gridded sub-feature icon is mapped into a mark feature, and all gridded sub-feature graphs form a mark feature sequence
X i Denotes the ith marker feature, i =1, 2.. N is the serial number of the marker feature.
In a further refinement, said step (5) further comprises upsampling aggregation and hopping chaining, comprising the steps of:
(5.1) firstly, carrying out layer normalization processing on the marked feature sequences,
x is a marked characteristic sequence after layer normalization processing,
the marked characteristic sequence obtained in the step (4);
(5.2) firstly carrying out frequency decomposition on each marking feature in the marking feature sequence in the previous step to decompose the marking features into high-frequency marking features and low-frequency marking features;
(5.3) for high frequency features, first performing a max pooling operation, then passing through a linear layer, and then passing through a depth separable convolutional layer, the pooling layer, the linear layer, and the depth separable convolutional layer forming a high frequency feature processor, the output being Y high ＝DConv(FC(MaxPool(X hihg ) )) in which Y is high For feature output after high frequency feature processor processing, DConv is deep separable convolution processing, FC is linear full join processing, maxPool is maximum pooling processing, X hihg Is the high frequency signature obtained from the signature decomposition in step (5.2);
(5.4) for the low-frequency marking features, firstly carrying out average pooling operation, then passing through an attention-seeking mechanism layer MSA, and then carrying out up-sampling to make up for the dimension reduction after the average pooling operation, wherein the average pooling layer, the attention-seeking mechanism layer and the up-sampling layer form a low-frequency feature positionPhysical processor with output of Y low ＝Upsample(MSA(AvePool(X low ) )) in which Y is low Is the feature output after processing by the low frequency feature processor, upesple is the upsampling process, MSA is the self attention mechanism process, avePool is the averaging pooling process, X low Is the low frequency signature feature obtained from the signature feature decomposition in step (5.2) above;
(5.5) high-low frequency feature fusion, namely, the high-frequency feature output and the low-frequency feature output are fused to obtain fused output Y o ＝Concat(Y high ，Y low ) In which Y is o For the fused feature output, concat is a feature linking function;
(5.6) carrying out layer normalization operation on the fused feature output again;
(5.7) after normalization operation, outputting the final BiFTrans block module through a feed-forward network FFN
Wherein
Is the signature sequence of step (4), Y o The characteristic output is obtained after high-frequency and low-frequency fusion, FFN is feedforward network processing, and Norm is layer normalization processing.
The invention has the beneficial effects that: according to the invention, through the linear transformation of GhostNet, a feature graph required by feature mining is generated by using simple linear transformation, so that network parameters are reduced, the complexity is reduced, the subsequent segmentation processing efficiency is ensured, and the response speed is improved; the method has the advantages that the high-Frequency and low-Frequency division processing characteristics are realized by using a double-Frequency mixer arranged in a Bi-Frequency Transformer (BiFTrans block) module, the high-Frequency details are captured, meanwhile, the low-Frequency global information is considered, the speed and the processing stability of image processing are ensured, the segmentation network of the GhostNet network is fused after the high-Frequency and low-Frequency processing characteristics, the comprehensive characteristics of the high-Frequency and low-Frequency information in the garbage image data are effectively learned by the segmentation network, the accuracy and the precision of the garbage target segmentation with serious deformation and pollution and fuzzy boundary sense are improved, and the subsequent classification efficiency is improved.
The invention is further described with reference to the following detailed description and accompanying drawings.
Drawings
Fig. 1 is a schematic flow chart of a garbage segmentation method based on a GinTrans network according to this embodiment;
FIG. 2 is a schematic diagram of the GinTrans network structure of the present embodiment;
fig. 3 is a schematic structural diagram of the Ghost module in this embodiment;
fig. 4 is a schematic structural diagram of the bifrans block module in this embodiment;
fig. 5 is a schematic structural diagram of the dual-frequency mixer of the present embodiment.
Detailed Description
The following description is only a preferred embodiment of the present invention, and does not limit the scope of the present invention.
In an embodiment, referring to fig. 1 to 5, a method for garbage segmentation based on a GinTrans network includes the following steps:
(1) Image segmentation system settings: the image segmentation system comprises an image acquisition module, an extraction module, a cutting identifier recombination module, a remodeling fusion output module, a jump connection aggregation module and a segmentation output module, wherein the extraction module is provided with a GhostNet network, the remodeling fusion output module is provided with a Bi-Frequency Transformer (BiFTrans block) module, the jump connection aggregation module is provided with an upper sampling recovery module and a GhostNet down-sampling feature extraction module, and the segmentation output module is provided with a segmentation head module;
(2) Image acquisition input: the intelligent garbage collection and sorting line comprises an image collection module, an image collection module and a lifting module, wherein the image collection module is connected with an image collection camera, the image collection camera collects images of garbage on the intelligent garbage collection and sorting line and transmits the images to the image collection module, and the images collected by the image collection module are used as image input data and are transmitted to the lifting module for processing;
(3) Extracting a characteristic diagram: the extraction module processes the image input data, and the extraction module adopts a GhostNet network to extract bottom layer features of the image input data, extracts an output feature map and carries out the next step of processing;
(4) Cutting identification and recombination: the cutting identification recombination module is used for cutting the extracted output characteristic diagram, the output characteristic diagram is gridded through the cutting operation, marking characteristics are generated through linear mapping after the gridding is finished, the marking characteristics form a marking characteristic sequence, and the marking characteristic sequence is used as the input of a Bi-Frequency Transformer (BiFTrans block) module;
(5) Remodeling and fusion: the remodeling fusion output module remodels the input marked feature sequence, and utilizes a dual-Frequency mixer arranged in a Bi-Frequency transform (BiFTrans block) module to perform fusion action on the features from different high and low frequencies, and the fusion action obtains fusion feature output;
(6) Jumping aggregation: recovering the fused feature output into an input image with the same size as the image input data through an up-sampling recovery module, extracting the lower material features of the corresponding same-layer resolution feature map through a GhostNet down-sampling feature extraction module, and jumping and linking the input image obtained by the up-sampling recovery module and the same-layer resolution feature map obtained by the GhostNet down-sampling feature extraction module so as to aggregate feature images presented in different resolution levels;
(7) And (3) segmentation output: and the segmentation head module of the segmentation output module generates and outputs the feature image to obtain a target image.
The improvement is further that the GhostNet network is provided with a Ghost module, and the design of the Ghost module is as follows:
(3.1) carrying out convolution on the input data of the to-be-identified garbage image in the step (2) through a GhostNet network to generate original feature maps f with different resolutions i ′；
(3.2) the extraction module contains linear operation phi, and the linear operation phi is utilized to generate a Ghost characteristic diagram as an output characteristic diagram, wherein the formula is f ij ＝Φ i，j (f i ') wherein, f i ' is the ith original characteristic map, phi, obtained by convolution operation of the garbage image to be identified i，j Is to the jth of the ith original feature mapLinear operation is carried out, and the jth Ghost characteristic diagram f is obtained after the operation ij 。
In a further improvement, the step (4) further comprises the following steps:
(4.1) performing gridding cutting on each Ghost feature map output in the step (3), wherein the size of each feature map is H multiplied by W, the H and the W are respectively the length and the width of the feature map, the size of each grid sub feature map is s multiplied by s, the s is the side length of the grid, and the s can be divided by the H and the W into grid sub feature maps;
(4.2) carrying out linear mapping on the gridded sub-feature graphs, wherein each gridded sub-feature icon is mapped into a mark feature, and all gridded sub-feature graphs form a mark feature sequence
X i Denotes the ith marker feature, i =1, 2.. N is the serial number of the marker feature.
In a further refinement, said step (5) further comprises upsampling aggregation and hopping chaining, comprising the steps of:
(5.1) firstly, carrying out layer normalization processing on the marked feature sequences,
x is a marked characteristic sequence after layer normalization processing,
the marked characteristic sequence obtained in the step (4);
(5.2) firstly carrying out frequency decomposition on each marking feature in the marking feature sequence in the previous step to decompose the marking features into high-frequency marking features and low-frequency marking features;
(5.3) for the high frequency signature, first performing a maximum pooling operation, then passing through a linear layer, and then passing through a depth-separable convolutional layer, the pooling layer, the linear layer, and the depth-separable convolutional layer forming a high frequency signature processor with an output of Y high ＝DConv(FC(MaxPool(X hihg ) In which Y) is present, wherein Y is high For the processed feature output of the high frequency feature processor, DConv is the deep separable convolution process, FC is the linear full join process, maxpool is the maximum pooling process, X hihg Is the high frequency signature obtained from the signature decomposition in step (5.2);
(5.4) for the low-frequency marking features, firstly carrying out average pooling operation, then passing through an attention-seeking mechanism layer MSA, and then carrying out up-sampling to make up for the dimension reduction after the average pooling operation, wherein the average pooling layer, the attention-seeking mechanism layer and the up-sampling layer form a low-frequency feature processor, and the output is Y low ＝Upsample(MSA(AvePool(X low ) )) in which Y is low Is the feature output after processing by the low frequency feature processor, upesple is the upsampling process, MSA is the self attention mechanism process, avePool is the averaging pooling process, X low Is the low frequency signature feature obtained from the signature feature decomposition in step (5.2) above;
(5.5) high-low frequency feature fusion, namely, fusing the high-frequency feature output and the low-frequency feature output to obtain fused output Y o ＝Concat(Y high ，Y low ) Wherein Y is o For the fused feature output, concat is a feature linking function;
(5.6) carrying out layer normalization operation on the fused feature output again;
(5.7) after normalization operation, outputting the final BiFTrans block module through a feed-forward network FFN
Wherein
Is the signature sequence of step (4), Y o The characteristic output is obtained after high-frequency and low-frequency fusion, FFN is feedforward network processing, and Norm is layer normalization processing.
According to the invention, through the linear transformation of GhostNet, a feature graph required by feature mining is generated by using simple linear transformation, so that network parameters are reduced, the complexity is reduced, the subsequent segmentation processing efficiency is ensured, and the response speed is improved; the method has the advantages that the high-Frequency and low-Frequency division processing characteristics are realized by using a double-Frequency mixer arranged in a Bi-Frequency Transformer (BiFTrans block) module, the high-Frequency details are captured, meanwhile, the low-Frequency global information is considered, the speed and the processing stability of image processing are ensured, the segmentation network of the GhostNet network is fused after the high-Frequency and low-Frequency processing characteristics, the comprehensive characteristics of the high-Frequency and low-Frequency information in the garbage image data are effectively learned by the segmentation network, the accuracy and the precision of the garbage target segmentation with serious deformation and pollution and fuzzy boundary sense are improved, and the subsequent classification efficiency is improved.
The present invention is not limited to the above embodiments, and other methods and systems for garbage segmentation based on the GinTrans network, which are obtained by using the same or similar structures, devices, processes or methods as those of the above embodiments of the present invention, are within the scope of the present invention.
Claims (4)
1. A garbage segmentation method based on a GinTrans network is characterized by comprising the following steps:
(1) The image segmentation system is set up as follows: the image segmentation system comprises an image acquisition module, an extraction module, a cutting identifier recombination module, a remodeling fusion output module, a jump connection aggregation module and a segmentation output module, wherein the extraction module is provided with a GhostNet network, the remodeling fusion output module is provided with a Bi-Frequency Transformer (BiFTrans block) module, the jump connection aggregation module is provided with an upper sampling recovery module and a GhostNet down-sampling feature extraction module, and the segmentation output module is provided with a segmentation head module;
(2) Image acquisition and input: the image acquisition module is connected with an image acquisition camera, the image acquisition camera acquires images of the garbage on the intelligent garbage recycling and sorting line and transmits the images to the image acquisition module, and the images acquired by the image acquisition module are used as image input data and are transmitted to the extraction module for processing;
(3) Extracting a characteristic diagram: the extraction module processes the image input data, and the extraction module adopts a GhostNet network to extract bottom layer features of the image input data, extracts an output feature map and carries out the next step of processing;
(4) Cutting identification and recombination: the cutting identification recombination module is used for cutting the extracted output characteristic diagram, the output characteristic diagram is gridded through the cutting operation, marking characteristics are generated through linear mapping after the gridding is finished, the marking characteristics form a marking characteristic sequence, and the marking characteristic sequence is used as the input of a Bi-Frequency Transformer (BiFTrans block) module;
(5) Remodeling and fusion: the remodeling fusion output module remodels the input marked feature sequence, and utilizes a dual-Frequency mixer arranged in a Bi-Frequency transform (BiFTrans block) module to perform fusion action on the features from different high and low frequencies, and the fusion action obtains fusion feature output;
(6) Jumping aggregation: recovering the fused feature output into an input image with the same size as the image input data through an up-sampling recovery module, extracting the lower material features of the corresponding same-layer resolution feature map through a GhostNet down-sampling feature extraction module, and jumping and linking the input image obtained by the up-sampling recovery module and the same-layer resolution feature map obtained by the GhostNet down-sampling feature extraction module so as to aggregate feature images presented in different resolution levels;
(7) And (3) segmentation output: and the segmentation head module of the segmentation output module generates segmentation output for the characteristic image to obtain a target image.
2. The GinTrans network-based garbage segmentation method according to claim 1, wherein the GhostNet network is provided with a Ghost module, and the Ghost module is designed as follows:
(3.1) carrying out convolution on the input data of the garbage image to be identified in the step (2) through a GhostNet network to generate original feature maps f with different resolutions i ′；
(3.2) the extraction module contains linear operation phi, and the linear operation phi is utilized to generate a Ghost characteristic diagram as an output characteristic diagram, wherein the formula is f ij ＝Φ i，j (f′ i ) Wherein f' i Is the ith original characteristic diagram phi obtained by convolution operation of the garbage image to be identified i，j The j th linear operation is carried out on the ith original feature map, and the j th Ghost feature map f is obtained after the operation ij 。
3. The GinTrans network-based garbage segmentation method according to claim 2, wherein the step (4) further comprises the steps of:
(4.1) performing gridding cutting on each Ghost feature map output in the step (3), wherein the size of each feature map is H multiplied by W, the H and the W are respectively the length and the width of the feature map, the size of each grid sub feature map is s multiplied by s, the s is the side length of the grid, and the s can be divided by the H and the W into grid sub feature maps;
(4.2) carrying out linear mapping on the gridded sub-feature graphs, wherein each gridded sub-feature icon is mapped into a mark feature, and all gridded sub-feature graphs form a mark feature sequence
Xi denotes the ith marker feature, i =1,2,. N is the serial number of the marker feature.
4. A GinTrans network based spam splitting method according to claim 3, wherein the step (5) further comprises upsampling aggregation and hopping chaining, comprising the steps of:
(5.1) firstly, the marked characteristic sequence is subjected to layer normalization processing,
x is the marked characteristic sequence after the layer normalization processing,
the marked characteristic sequence obtained in the step (4);
(5.2) firstly carrying out frequency decomposition on each marking feature in the marking feature sequence in the previous step to decompose the marking features into high-frequency marking features and low-frequency marking features;
(5.3) for high frequency features, maximum pooling is performed firstOperating, then passing through a linear layer, then through a depth separable convolutional layer, the pooling layer, the linear layer and the depth separable convolutional layer forming a high frequency feature processor with an output of Y high ＝DConv(FC(MaxPool(X hihg ) In which Y) is present, wherein Y is high For feature output after high frequency feature processor processing, DConv is deep separable convolution processing, FC is linear full join processing, maxPool is maximum pooling processing, X hihg Is the high frequency signature obtained from the signature decomposition in step (5.2);
(5.4) for the low-frequency marking features, firstly carrying out average pooling operation, then passing through an attention-seeking mechanism layer MSA, and then carrying out up-sampling to make up for the dimension reduction after the average pooling operation, wherein the average pooling layer, the attention-seeking mechanism layer and the up-sampling layer form a low-frequency feature processor, and the output is Y low ＝Upsample(MSA(AvePool(X low ) In which Y) is present, wherein Y is low Is the feature output after processing by the low frequency feature processor, upsample is the upsampling process, MSA is the autofocusing mechanism process, avePool is the averaging pooling process, X low The low-frequency marking features are obtained by the marking feature decomposition in the step (5.2);
(5.5) high-low frequency feature fusion, namely, the high-frequency feature output and the low-frequency feature output are fused to obtain fused output Y o ＝Concat(Y high ，Y low ) Wherein Y is o For the fused feature output, concat is a feature linking function;
(5.6) carrying out layer normalization operation on the fused feature output again;
(5.7) after normalization operation, outputting the final BiFTrans block module through a feed-forward network FFN
Wherein
Is the signature sequence of step (4), Y O Is the characteristic output after high and low frequency fusion, FFN is feedforward network processing, norm is layer normalization processing。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210901322.8A CN115147703B (en) | 2022-07-28 | 2022-07-28 | Garbage segmentation method and system based on GinTrans network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210901322.8A CN115147703B (en) | 2022-07-28 | 2022-07-28 | Garbage segmentation method and system based on GinTrans network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115147703A true CN115147703A (en) | 2022-10-04 |
| CN115147703B CN115147703B (en) | 2023-11-03 |
Family
ID=83413245
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210901322.8A Active CN115147703B (en) | 2022-07-28 | 2022-07-28 | Garbage segmentation method and system based on GinTrans network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115147703B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6381363B1 (en) * | 1999-03-15 | 2002-04-30 | Grass Valley (U.S.), Inc. | Histogram-based segmentation of images and video via color moments |
| CN112017191A (en) * | 2020-08-12 | 2020-12-01 | 西北大学 | Method for establishing and segmenting liver pathology image segmentation model based on attention mechanism |
| CN112541503A (en) * | 2020-12-11 | 2021-03-23 | 南京邮电大学 | Real-time semantic segmentation method based on context attention mechanism and information fusion |
| WO2021115483A1 (en) * | 2019-12-13 | 2021-06-17 | 华为技术有限公司 | Image processing method and related apparatus |
| CN113051430A (en) * | 2021-03-26 | 2021-06-29 | 北京达佳互联信息技术有限公司 | Model training method, device, electronic equipment, medium and product |
| CN113102266A (en) * | 2021-03-16 | 2021-07-13 | 四川九通智路科技有限公司 | Multi-dimensional garbage recognition and classification system |
| US20210224512A1 (en) * | 2020-01-17 | 2021-07-22 | Wuyi University | Danet-based drone patrol and inspection system for coastline floating garbage |
| CN113159051A (en) * | 2021-04-27 | 2021-07-23 | 长春理工大学 | Remote sensing image lightweight semantic segmentation method based on edge decoupling |
| CN113591939A (en) * | 2021-07-09 | 2021-11-02 | 上海智臻智能网络科技股份有限公司 | Layer classification method and device |
| CN113744292A (en) * | 2021-09-16 | 2021-12-03 | 安徽世绿环保科技有限公司 | Garbage classification station garbage throwing scanning system |
| US11222217B1 (en) * | 2020-08-14 | 2022-01-11 | Tsinghua University | Detection method using fusion network based on attention mechanism, and terminal device |
| CN114708464A (en) * | 2022-06-01 | 2022-07-05 | 广东艺林绿化工程有限公司 | Municipal sanitation cleaning garbage truck cleaning method based on road garbage classification |
| WO2022141723A1 (en) * | 2020-12-29 | 2022-07-07 | 江苏大学 | Image classification and segmentation apparatus and method based on feature guided network, and device and medium |
-
2022
- 2022-07-28 CN CN202210901322.8A patent/CN115147703B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6381363B1 (en) * | 1999-03-15 | 2002-04-30 | Grass Valley (U.S.), Inc. | Histogram-based segmentation of images and video via color moments |
| WO2021115483A1 (en) * | 2019-12-13 | 2021-06-17 | 华为技术有限公司 | Image processing method and related apparatus |
| US20210224512A1 (en) * | 2020-01-17 | 2021-07-22 | Wuyi University | Danet-based drone patrol and inspection system for coastline floating garbage |
| CN112017191A (en) * | 2020-08-12 | 2020-12-01 | 西北大学 | Method for establishing and segmenting liver pathology image segmentation model based on attention mechanism |
| US11222217B1 (en) * | 2020-08-14 | 2022-01-11 | Tsinghua University | Detection method using fusion network based on attention mechanism, and terminal device |
| CN112541503A (en) * | 2020-12-11 | 2021-03-23 | 南京邮电大学 | Real-time semantic segmentation method based on context attention mechanism and information fusion |
| WO2022141723A1 (en) * | 2020-12-29 | 2022-07-07 | 江苏大学 | Image classification and segmentation apparatus and method based on feature guided network, and device and medium |
| CN113102266A (en) * | 2021-03-16 | 2021-07-13 | 四川九通智路科技有限公司 | Multi-dimensional garbage recognition and classification system |
| CN113051430A (en) * | 2021-03-26 | 2021-06-29 | 北京达佳互联信息技术有限公司 | Model training method, device, electronic equipment, medium and product |
| CN113159051A (en) * | 2021-04-27 | 2021-07-23 | 长春理工大学 | Remote sensing image lightweight semantic segmentation method based on edge decoupling |
| CN113591939A (en) * | 2021-07-09 | 2021-11-02 | 上海智臻智能网络科技股份有限公司 | Layer classification method and device |
| CN113744292A (en) * | 2021-09-16 | 2021-12-03 | 安徽世绿环保科技有限公司 | Garbage classification station garbage throwing scanning system |
| CN114708464A (en) * | 2022-06-01 | 2022-07-05 | 广东艺林绿化工程有限公司 | Municipal sanitation cleaning garbage truck cleaning method based on road garbage classification |
Non-Patent Citations (2)
| Title |
|---|
| ZHU X ET AL: "Deformable transformers for end-to-end object detection", 《ARXIV》 * |
| 刘小玉等: "基于DeeplabV3+的建筑垃圾堆放点识别", 《测绘通报》, no. 04 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115147703B (en) | 2023-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106877237B (en) | A method of insulator lacks in the detection transmission line of electricity based on Aerial Images | |
| CN103824309B (en) | Automatic extracting method of urban built-up area border | |
| CN106971152A (en) | A kind of method of Bird's Nest in detection transmission line of electricity based on Aerial Images | |
| CN105718912B (en) | A kind of vehicle characteristics object detecting method based on deep learning | |
| CN111368895A (en) | Garbage bag target detection method and detection system in wet garbage | |
| CN104486585B (en) | A kind of city magnanimity monitor video management method and system based on GIS | |
| CN110188802B (en) | SSD target detection algorithm based on multi-layer feature map fusion | |
| CN111597983B (en) | Method for realizing identification of generated false face image based on deep convolutional neural network | |
| CN109003275A (en) | The dividing method of weld defect image | |
| CN112364944B (en) | Deep learning-based household garbage classification method | |
| CN105261006A (en) | Medical image segmentation algorithm based on Fourier transform | |
| CN111008979A (en) | Robust night image semantic segmentation method | |
| Meng et al. | X-DenseNet: deep learning for garbage classification based on visual images | |
| CN112800982A (en) | Target detection method based on remote sensing scene classification | |
| CN111461000A (en) | Intelligent office garbage classification method based on CNN and wavelet analysis | |
| CN111797787B (en) | Waste image detection and classification system based on Internet of things technology | |
| CN110683240A (en) | Garbage classification processing system based on image processing | |
| CN111680640B (en) | Vehicle type identification method and system based on domain migration | |
| CN115147703A (en) | GinTrans network-based garbage segmentation method and system | |
| CN107367514A (en) | A kind of POS defect inspection method based on machine vision | |
| CN116152571A (en) | Kitchen waste identification and classification method based on deep learning | |
| CN101527001A (en) | Secret information detecting system based on expert system method | |
| CN113516114B (en) | Natural scene text detection method, equipment and medium | |
| CN109697693A (en) | A method of it realizes based on big data spatial operation | |
| CN112992304B (en) | High-resolution red eye case data generation method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |