CN112435179A - Fuzzy pollen particle picture processing method and device and electronic equipment - Google Patents
Fuzzy pollen particle picture processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN112435179A CN112435179A CN202011255617.XA CN202011255617A CN112435179A CN 112435179 A CN112435179 A CN 112435179A CN 202011255617 A CN202011255617 A CN 202011255617A CN 112435179 A CN112435179 A CN 112435179A
- Authority
- CN
- China
- Prior art keywords
- pollen particle
- fuzzy
- clear
- pollen
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000002245 particle Substances 0.000 title claims abstract description 394
- 238000003672 processing method Methods 0.000 title claims abstract description 17
- 238000012549 training Methods 0.000 claims abstract description 55
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000009826 distribution Methods 0.000 claims abstract description 17
- 238000010586 diagram Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 8
- 238000003860 storage Methods 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 5
- 238000005520 cutting process Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 13
- 238000010276 construction Methods 0.000 abstract description 7
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 230000000172 allergic effect Effects 0.000 description 3
- 208000010668 atopic eczema Diseases 0.000 description 3
- 238000000605 extraction Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- IJJWOSAXNHWBPR-HUBLWGQQSA-N 5-[(3as,4s,6ar)-2-oxo-1,3,3a,4,6,6a-hexahydrothieno[3,4-d]imidazol-4-yl]-n-(6-hydrazinyl-6-oxohexyl)pentanamide Chemical compound N1C(=O)N[C@@H]2[C@H](CCCCC(=O)NCCCCCC(=O)NN)SC[C@@H]21 IJJWOSAXNHWBPR-HUBLWGQQSA-N 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/001—Model-based coding, e.g. wire frame
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a fuzzy pollen particle picture processing method, a fuzzy pollen particle picture processing device and electronic equipment, wherein a to-be-processed fuzzy pollen particle picture is input to generate an confrontation network model, and a clear pollen particle picture is output; the method comprises the steps that a confrontation network model is generated by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data and conducting unsupervised training; the clear pollen particle picture corresponding to the fuzzy pollen particle picture is a clear pollen particle picture with the largest feature similarity with the fuzzy pollen particle picture in different classes of clear pollen particle pictures. The method saves the construction cost of the real paired data, relieves the difficulty that the paired data are difficult to obtain, effectively avoids the inconsistency of data distribution between the synthetic data and the real data, and improves the deblurring effect of the model on the real fuzzy data.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a fuzzy pollen particle image processing method and device and electronic equipment.
Background
In recent years, the expansion of plant growing areas during urbanization construction has become an indispensable part of urbanization construction. However, as the plant growing area expands, the pollen of a large number of sensitized plants floating in the air can greatly increase the incidence of pollen allergic patients, which can seriously affect the daily life of the allergic patients. However, effective pollen concentration prediction can greatly reduce the incidence of allergic patients. However, the important premise for effectively forecasting the pollen concentration is that the pollen particles can be accurately identified, and the clear pollen particle picture is the basis for identifying the pollen particles. Therefore, the task of deblurring the pollen particle picture becomes more important.
The pollen particle picture deblurring belongs to picture restoration, most of current image deblurring algorithms need paired data to train a model, but the acquisition cost of the paired data is high, and in many cases, the paired data does not exist, so that the general solution of the algorithm is to adopt a method of synthesizing data pairs to construct the paired data for model training.
However, while some solutions to the problem of pair-wise data unavailability are provided by deblurring algorithms that employ synthetic data pairs, such algorithms do not perform satisfactorily on real data. The reason is that the data distribution difference exists between the synthetic data and the real data, and the difference causes that the final training result of the model only better fits the data distribution of the synthetic data, but the fitting effect on the real data is poor, so that the deblurring effect of the model on the real fuzzy data is poor.
Disclosure of Invention
Aiming at the technical problems in the prior art, the invention provides a fuzzy pollen particle picture processing method and device and electronic equipment.
In a first aspect, the present invention provides a fuzzy pollen particle picture processing method, including:
inputting the fuzzy pollen particle picture to be processed into a confrontation network model, and outputting a clear pollen particle picture;
the generated confrontation network model is obtained by carrying out unsupervised training by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data; the clear pollen particle pictures corresponding to the fuzzy pollen particle pictures are clear pollen particle pictures with the largest feature similarity with the fuzzy pollen particle pictures in different classes of clear pollen particle pictures.
Optionally, the clear pollen particle picture corresponding to the blurred pollen particle picture is a clear pollen particle picture with the largest feature similarity with the blurred pollen particle picture in different categories of clear pollen particle pictures, and the clear pollen particle picture includes:
inputting the different types of clear pollen particle pictures into an encoder, and outputting the feature maps of the different types of clear pollen particle pictures;
inputting the fuzzy pollen particle picture into an encoder, and outputting a characteristic diagram of the fuzzy pollen particle picture;
calculating the feature similarity between the feature maps of the different classes of clear pollen particle pictures and the feature map of the fuzzy pollen particle picture;
and determining the clear pollen particle picture corresponding to the feature picture of the clear pollen particle picture with the maximum feature similarity between the feature pictures of the fuzzy pollen particle picture as the clear pollen particle picture corresponding to the fuzzy pollen particle picture.
Optionally, the generating a countermeasure network model comprises a generator and a discriminator;
the generated confrontation network model is obtained by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data and performing unsupervised training, and comprises the following steps:
the fuzzy pollen particle picture is used for training the generator, and the clear pollen particle picture corresponding to the fuzzy pollen particle picture is used for training the discriminator;
the generator generates a clear pollen particle picture based on the fuzzy pollen particle picture, the discriminator discriminates whether the clear pollen particle picture generated by the generator is accurate or not based on the clear pollen particle picture corresponding to the fuzzy pollen particle picture, and if so, the training is finished; if not, continuing training.
Optionally, the fuzzy pollen particle picture is used for training the generator, and the clear pollen particle picture corresponding to the fuzzy pollen particle picture is used for training the discriminator, specifically:
LD＝Es～p(s)[log D(s)]+Eb～p(b)[log(1-D(G(E(b))))]；
wherein L isDRepresenting a loss function of a discriminator, D representing the discriminator, G representing a generator, E representing an encoder, s representing a clear pollen particle picture with the highest feature similarity with the fuzzy pollen particle picture, b representing the fuzzy pollen particle picture, and p(s) and p (b) respectively representing the probability distribution of the clear pollen particle picture and the probability distribution of the fuzzy pollen particle picture.
Optionally, the calculating the feature similarity between the feature maps of the different types of clear pollen particle pictures and the feature map of the fuzzy pollen particle picture specifically includes:
wherein L issRepresenting a value of a characteristic difference, Φ, between the blurred and the clear pollen grain picturesi(s) feature map of the ith layer of the clear pollen grain picture,. phii(b) Feature map, M, representing the ith layer of a blurred pollen grain pictureiIndicating the size of the ith layer signature.
Optionally, before inputting the different categories of clear pollen particle pictures into the encoder, the method further includes:
cutting the different types of clear pollen particle pictures;
and randomly and horizontally rotating the clear pollen particle pictures of the categories with the number smaller than the preset value.
In a second aspect, the present invention provides a fuzzy pollen particle picture processing device, comprising:
the input module is used for inputting the fuzzy pollen particle picture to be processed to generate a confrontation network model;
the output module is used for outputting a clear pollen particle picture;
the generated confrontation network model is obtained by carrying out unsupervised training by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data; the clear pollen particle pictures corresponding to the fuzzy pollen particle pictures are clear pollen particle pictures with the largest feature similarity with the fuzzy pollen particle pictures in different categories of clear pollen particle pictures.
Optionally, the clear pollen particle picture corresponding to the fuzzy pollen particle picture is a clear pollen particle picture with the largest feature similarity with the fuzzy pollen particle picture in different categories of clear pollen particle pictures, and the clear pollen particle picture includes:
inputting the different types of clear pollen particle pictures into an encoder, and outputting the feature maps of the different types of clear pollen particle pictures;
inputting the fuzzy pollen particle picture into an encoder, and outputting a characteristic diagram of the fuzzy pollen particle picture;
calculating the feature similarity between the feature maps of the different classes of clear pollen particle pictures and the feature map of the fuzzy pollen particle picture;
determining that the clear pollen particle picture corresponding to the feature graph of the clear pollen particle picture with the maximum feature similarity between the feature graphs of the fuzzy pollen particle pictures is the clear pollen particle picture corresponding to the fuzzy pollen particle picture
In a third aspect, the present invention provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method as provided in the first aspect when executing the program.
In a fourth aspect, the invention provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method as provided in the first aspect.
The fuzzy pollen particle picture processing method, the fuzzy pollen particle picture processing device and the electronic equipment form a pseudo data pair with the clear pollen particle picture with the largest characteristic similarity with the fuzzy pollen particle picture, and are used for training to generate an confrontation network model so as to achieve deblurring processing of the fuzzy pollen particle picture, save construction cost of real paired data, relieve the problem that the paired data is difficult to obtain, effectively avoid inconsistency of data distribution between synthetic data and the real data, and improve deblurring effect of the model on the real fuzzy data.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a fuzzy pollen particle picture processing method provided by the present invention;
FIG. 2 is a schematic diagram of the structure of the generative confrontation model provided by the present invention;
FIG. 3 is a schematic structural diagram of a fuzzy pollen particle picture processing device provided by the present invention;
fig. 4 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the prior art, a synthetic data pair is usually adopted to train an image deblurring model, but due to the data distribution difference existing between the synthetic data and the real data, the fitting effect of the final training result of the model on the real data is poor, and the deblurring effect of the model is poor.
Therefore, the invention provides a fuzzy pollen particle picture processing method. Fig. 1 is a schematic flow chart of a fuzzy pollen particle image processing method provided by the present invention, as shown in fig. 1, the method includes:
s101: inputting a fuzzy pollen particle picture to be processed to generate a confrontation network model;
s102: outputting a clear pollen particle picture;
the generated confrontation network model is obtained by carrying out unsupervised training by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data; the clear pollen particle pictures corresponding to the fuzzy pollen particle pictures are clear pollen particle pictures with the largest feature similarity with the fuzzy pollen particle pictures in different classes of clear pollen particle pictures.
Specifically, because the characteristics of the fuzzy pollen particle pictures and the characteristics of the existing clear pollen particle pictures have certain similarity, the clear pollen particle pictures with the maximum characteristic similarity to the fuzzy pollen particle pictures in the clear pollen particle pictures of different types are selected as the clear pollen particle pictures corresponding to the fuzzy pollen particle pictures to form a pseudo data pair by calculating the characteristic similarity between the clear pollen particle pictures of different types and the fuzzy pollen particle pictures, and the pseudo data pair is used as sample data to perform unsupervised training on a countermeasure network model to obtain a generated countermeasure network model with a good deblurring effect after training, and the generated countermeasure network model after training is used for completing the deblurring processing of the fuzzy pollen particle pictures: and inputting the fuzzy pollen particle picture to be processed into the trained confrontation network model, and outputting a clear pollen particle picture.
According to the method, the fuzzy pollen particle picture and the clear pollen particle picture with the largest characteristic similarity with the fuzzy pollen particle picture form a pseudo data pair for training to generate a confrontation network model so as to achieve deblurring processing on the fuzzy pollen particle picture, save construction cost of real paired data, relieve the difficulty that the paired data is difficult to obtain, effectively avoid inconsistency of data distribution between synthetic data and the real data, and improve the deblurring effect of the model on the real fuzzy data.
Based on the above embodiment, the clear pollen particle picture corresponding to the blurred pollen particle picture is a clear pollen particle picture with the largest feature similarity with the blurred pollen particle picture in different categories of clear pollen particle pictures, and the clear pollen particle picture includes:
inputting the different types of clear pollen particle pictures into an encoder, and outputting the feature maps of the different types of clear pollen particle pictures;
and inputting the fuzzy pollen particle picture into an encoder, and outputting a characteristic diagram of the fuzzy pollen particle picture.
Specifically, the processed different types of clear pollen particle pictures are input into an encoder, and feature maps of the different types of clear pollen particle pictures are output. As shown in fig. 2, the Encoder (Encoder) includes 4 volume blocks and 2 max-pooling layers, wherein each volume block includes three parts: 64 convolution kernels of 3X3, a Bach Normalization operation, and a Relu activation function. The clear pollen particle pictures of different categories input into the encoder are processed by 4 rolling blocks and 2 maximum pooling layers, and finally characteristic graphs of the clear pollen particle pictures of different categories are obtained; similarly, the fuzzy pollen particle picture is input into an encoder, and a characteristic diagram of the fuzzy pollen particle picture is output.
Calculating the feature similarity between the feature maps of the different classes of clear pollen particle pictures and the feature map of the fuzzy pollen particle picture;
and determining the clear pollen particle picture corresponding to the feature picture of the clear pollen particle picture with the maximum feature similarity between the feature pictures of the fuzzy pollen particle picture as the clear pollen particle picture corresponding to the fuzzy pollen particle picture.
Specifically, feature similarity between feature graphs of the different types of clear pollen particle pictures and feature graphs of the fuzzy pollen particle pictures is calculated, the clear pollen particle picture corresponding to the feature graph of the clear pollen particle picture with the maximum feature similarity to the feature graphs of the fuzzy pollen particle pictures is determined to be the clear pollen particle picture corresponding to the fuzzy pollen particle picture, namely the clear pollen particle picture closest to the fuzzy pollen particle picture, and the current fuzzy pollen particle picture and the determined clear pollen particle picture form a pseudo data pair for training to generate a confrontation network model.
Alternatively, the depth of the encoder network may be increased to further improve the accuracy of the feature similarity calculation result of the feature map of the blurred pollen particle picture, for example, a VGG19(Visual Geometry Group, super resolution test sequence) network is used to extract features of the pollen particle picture, so as to generate the feature map of the pollen particle picture.
The method of the invention generates the characteristic diagram of the clear pollen particle picture and the characteristic diagram of the fuzzy pollen particle picture through the encoder, calculates the characteristic similarity between the characteristic diagram of the clear pollen particle picture and the characteristic diagram of the fuzzy pollen particle picture, determines the clear pollen particle picture corresponding to the fuzzy pollen particle picture, and forms the fuzzy pollen particle picture and the clear pollen particle picture corresponding to the fuzzy pollen particle picture into a pseudo data pair as a training sample for generating the confrontation network model, thereby saving the construction cost of real paired data, relieving the difficult problem that the paired data is difficult to obtain, effectively avoiding the inconsistency of data distribution between the synthetic data and the real data, and improving the deblurring effect of the model on the real fuzzy data.
According to any of the above embodiments, the generating the confrontation network model comprises a generator and an arbiter;
the generated confrontation network model is obtained by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data and performing unsupervised training, and comprises the following steps:
the fuzzy pollen particle picture is used for training the generator, and the clear pollen particle picture corresponding to the fuzzy pollen particle picture is used for training the discriminator;
the generator generates a clear pollen particle picture based on the fuzzy pollen particle picture, the discriminator discriminates whether the clear pollen particle picture generated by the generator is accurate or not based on the clear pollen particle picture corresponding to the fuzzy pollen particle picture, and if so, the training is finished; if not, continuing training.
Specifically, the generate countermeasure Networks (GAN) model module includes two parts: a Generator (Generator) and a Discriminator (Discriminator). As shown in fig. 2, the network structure of the generator includes operations such as density (), Tanh (), BN (), Conv2d (), Upsampling2D (), and the like, and aims to take the feature map of the blurred pollen particle picture extracted by the encoder as input, reconstruct the blurred picture through a series of transformation operations, and finally generate a deblurred pollen particle picture, that is, a clear pollen particle picture obtained through reconstruction; the network structure of the discriminator comprises operations of Conv2D (), Tanh (), MaxPooling2D (), Dense () and Sigmoid (), and the like, and aims to discriminate whether the picture generated by the generator is close to the input real clear pollen particle picture, if so, the model training is finished; if not, continuing training, and adjusting the parameters of the generator according to the comparison result of the image generated by the generator and the clear pollen particle image so that the image generated by the generator is closer to the clear pollen particle image.
Optionally, the generated countermeasure network model adopted by the invention is the original generated countermeasure network model, and the generated countermeasure network model used by the algorithm with better deblurring processing on the basis of the synthetic data pair as training data can also be adopted, so that the deblurring effect of the model is further improved.
According to the method, the fuzzy pollen particle picture and the clear pollen particle picture corresponding to the fuzzy pollen particle picture form a pseudo data pair to be used as a training sample for generating the confrontation network model, and the fuzzy picture processing effect of the confrontation network model is improved.
Based on any of the above embodiments, the fuzzy pollen particle picture is used for training the generator, and the clear pollen particle picture corresponding to the fuzzy pollen particle picture is used for training the discriminator, specifically:
LD＝Es～p(s)[logD(s)]+Eb～p(b)[log(1-D(G(E(b))))]；
wherein L isDRepresenting a loss function of a discriminator, D representing the discriminator, G representing a generator, E representing an encoder, s representing a clear pollen particle picture with the highest feature similarity with the fuzzy pollen particle picture, b representing the fuzzy pollen particle picture, and p(s) and p (b) respectively representing the probability distribution of the clear pollen particle picture and the probability distribution of the fuzzy pollen particle picture.
Specifically, in the training process, clear pollen particle pictures in the pseudo data pairs are adopted to train a discriminator, and the final aim of the discriminator is to discriminate whether the pictures generated by the generator are close to the input real clear pollen particle pictures; and adopting the encoding vector of the fuzzy pollen particle picture in the pseudo data pair as the input of the generator, and inputting the generated result to the discriminator for discrimination. In the training phase, the generator is optimized by maximizing D (G (e (b)). The final aim of the generation is to generate a clear pollen particle picture close to the reality as far as possible, namely, the data distribution characteristics of the clear pollen particle picture are learned, so that the aim of deblurring the blurred pollen particle picture is fulfilled.
Based on any of the above embodiments, the calculating the feature similarity between the feature map of the different categories of clear pollen particle pictures and the feature map of the blurred pollen particle pictures specifically includes:
wherein L issRepresenting a value of a characteristic difference, Φ, between the blurred and the clear pollen grain picturesi(s) feature map of the ith layer of the clear pollen grain picture,. phii(b) Feature map, M, representing the ith layer of a blurred pollen grain pictureiIndicating the size of the ith layer signature.
In particular, LsA value representing the difference in characteristics, L, between the blurred and clear pollen grain picturessSmaller values of (a) indicate more similarity, whereas differences are greater. In the invention, the feature map of the last layer of the network is mainly selected to calculate the feature similarity.
According to any of the above embodiments, before inputting the different classes of clear pollen particle pictures into the encoder, the method further includes:
and cutting the different types of clear pollen particle pictures.
Specifically, since the pollen particle size is small and most pollen particles are located in the middle of the whole picture, for this purpose, before inputting different classes of clear pollen particle pictures into the encoder, the different classes of clear pollen particle pictures are cut to improve the efficiency and accuracy of feature extraction.
And randomly and horizontally rotating the clear pollen particle pictures of the categories with the number smaller than the preset value.
Specifically, the number of different classes of clear pollen particle picture samples may be unbalanced, for example, the number of some classes of clear pollen particle picture samples is large, and the number of other classes of clear pollen particle picture samples is small. The preset value may be set according to actual conditions, and the present invention is not limited thereto.
According to the method, before the different types of clear pollen particle pictures are input into the encoder, the different types of clear pollen particle pictures are cut, so that the efficiency and accuracy of feature extraction are improved; and randomly and horizontally rotating the clear pollen particle pictures of the categories with the number smaller than the preset value to construct more training samples and prevent the overfitting of the model.
The following describes the fuzzy pollen particle picture processing device provided by the present invention, and the fuzzy pollen particle picture processing device described below and the fuzzy pollen particle picture processing method described above can be referred to correspondingly.
Based on any of the above embodiments, fig. 3 is a schematic structural diagram of the fuzzy pollen particle picture processing apparatus provided by the present invention, as shown in fig. 3, the fuzzy pollen particle picture processing apparatus includes an input module 301 and an output module 302.
The input module 301 is configured to input a to-be-processed fuzzy pollen particle picture to generate a confrontation network model; the output module 302 is used for outputting a clear pollen particle picture; the generated confrontation network model is obtained by carrying out unsupervised training by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data; the clear pollen particle pictures corresponding to the fuzzy pollen particle pictures are clear pollen particle pictures with the largest feature similarity with the fuzzy pollen particle pictures in different categories of clear pollen particle pictures.
The device provided by the invention forms the fuzzy pollen particle picture and the clear pollen particle picture with the maximum characteristic similarity with the fuzzy pollen particle picture into a pseudo data pair for training and generating a confrontation network model so as to realize the deblurring treatment on the fuzzy pollen particle picture, save the construction cost of real paired data, relieve the difficult problem that the paired data is difficult to obtain, effectively avoid the inconsistency of data distribution between the synthetic data and the real data and improve the deblurring effect of the model on the real fuzzy data.
Based on any of the above embodiments, the clear pollen particle picture with the largest feature similarity with the fuzzy pollen particle picture in clear pollen particle pictures corresponding to the fuzzy pollen particle picture in different categories includes:
inputting the different types of clear pollen particle pictures into an encoder, and outputting the feature maps of the different types of clear pollen particle pictures;
inputting the fuzzy pollen particle picture into an encoder, and outputting a characteristic diagram of the fuzzy pollen particle picture;
calculating the feature similarity between the feature maps of the different classes of clear pollen particle pictures and the feature map of the fuzzy pollen particle picture;
and determining the clear pollen particle picture corresponding to the feature picture of the clear pollen particle picture with the maximum feature similarity between the feature pictures of the fuzzy pollen particle picture as the clear pollen particle picture corresponding to the fuzzy pollen particle picture.
According to any of the above embodiments, the generating the confrontation network model comprises a generator and an arbiter;
the generated confrontation network model is obtained by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data and performing unsupervised training, and comprises the following steps:
the fuzzy pollen particle picture is used for training the generator, and the clear pollen particle picture corresponding to the fuzzy pollen particle picture is used for training the discriminator;
the generator generates a clear pollen particle picture based on the fuzzy pollen particle picture, the discriminator discriminates whether the clear pollen particle picture generated by the generator is accurate or not based on the clear pollen particle picture corresponding to the fuzzy pollen particle picture, and if so, the training is finished; if not, continuing training.
Based on any of the above embodiments, the fuzzy pollen particle picture is used for training the generator, and the clear pollen particle picture corresponding to the fuzzy pollen particle picture is used for training the discriminator, specifically:
LD＝Es～p(s)[logD(s)]+Eb～p(b)[log(1-D(G(E(b))))]；
wherein L isDRepresenting a loss function of a discriminator, D representing the discriminator, G representing a generator, E representing an encoder, s representing a clear pollen particle picture with the highest feature similarity with the fuzzy pollen particle picture, b representing the fuzzy pollen particle picture, and p(s) and p (b) respectively representing the probability distribution of the clear pollen particle picture and the probability distribution of the fuzzy pollen particle picture.
Based on any of the above embodiments, the calculating the feature similarity between the feature map of the different categories of clear pollen particle pictures and the feature map of the blurred pollen particle pictures specifically includes:
wherein L issRepresenting a value of a characteristic difference, Φ, between the blurred and the clear pollen grain picturesi(s) feature map of the ith layer of the clear pollen grain picture,. phii(b) Feature map, M, representing the ith layer of a blurred pollen grain pictureiIndicating the size of the ith layer signature.
Based on any of the above embodiments, before the input module inputs the different categories of clear pollen particle pictures into the encoder, the method further includes:
cutting the different types of clear pollen particle pictures;
and randomly and horizontally rotating the clear pollen particle pictures of the categories with the number smaller than the preset value.
The fuzzy pollen particle picture processing device provided by the invention can be used for executing the technical scheme of each fuzzy pollen particle picture processing method embodiment, the implementation principle and the technical effect are similar, and the details are not repeated here.
Fig. 4 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 4: a processor (processor)410, a communication interface (communication interface)420, a memory (memory)430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 are communicated with each other via the communication bus 440. The processor 410 may invoke logic instructions in the memory 430 to perform a fuzzy pollen grain picture processing method comprising: inputting the fuzzy pollen particle picture to be processed into a confrontation network model, and outputting a clear pollen particle picture; the generated confrontation network model is obtained by carrying out unsupervised training by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data; the clear pollen particle pictures corresponding to the fuzzy pollen particle pictures are clear pollen particle pictures with the largest feature similarity with the fuzzy pollen particle pictures in different classes of clear pollen particle pictures.
In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to execute the fuzzy pollen particle picture processing method provided in the above, the method comprising: inputting the fuzzy pollen particle picture to be processed into a confrontation network model, and outputting a clear pollen particle picture; the generated confrontation network model is obtained by carrying out unsupervised training by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data; the clear pollen particle pictures corresponding to the fuzzy pollen particle pictures are clear pollen particle pictures with the largest feature similarity with the fuzzy pollen particle pictures in different classes of clear pollen particle pictures.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A fuzzy pollen particle picture processing method is characterized by comprising the following steps:
inputting the fuzzy pollen particle picture to be processed into a confrontation network model, and outputting a clear pollen particle picture;
the generated confrontation network model is obtained by carrying out unsupervised training by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data; the clear pollen particle pictures corresponding to the fuzzy pollen particle pictures are clear pollen particle pictures with the largest feature similarity with the fuzzy pollen particle pictures in different classes of clear pollen particle pictures.
2. The method for processing the blurred pollen particle picture as claimed in claim 1, wherein the clear pollen particle picture corresponding to the blurred pollen particle picture is a clear pollen particle picture with the largest feature similarity with the blurred pollen particle picture, among clear pollen particle pictures of different categories, and the method comprises:
inputting the different types of clear pollen particle pictures into an encoder, and outputting the feature maps of the different types of clear pollen particle pictures;
inputting the fuzzy pollen particle picture into an encoder, and outputting a characteristic diagram of the fuzzy pollen particle picture;
calculating the feature similarity between the feature maps of the different classes of clear pollen particle pictures and the feature map of the fuzzy pollen particle picture;
and determining the clear pollen particle picture corresponding to the feature picture of the clear pollen particle picture with the maximum feature similarity between the feature pictures of the fuzzy pollen particle picture as the clear pollen particle picture corresponding to the fuzzy pollen particle picture.
3. The fuzzy pollen grain picture processing method as claimed in claim 2, wherein said generating a confrontation network model comprises a generator and a discriminator;
the generated confrontation network model is obtained by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data and performing unsupervised training, and comprises the following steps:
the fuzzy pollen particle picture is used for training the generator, and the clear pollen particle picture corresponding to the fuzzy pollen particle picture is used for training the discriminator;
the generator generates a clear pollen particle picture based on the fuzzy pollen particle picture, the discriminator discriminates whether the clear pollen particle picture generated by the generator is accurate or not based on the clear pollen particle picture corresponding to the fuzzy pollen particle picture, and if so, the training is finished; if not, continuing training.
4. The method for processing the blurred pollen particle picture as claimed in claim 3, wherein the blurred pollen particle picture is used for training the generator, and the clear pollen particle picture corresponding to the blurred pollen particle picture is used for training the discriminator, specifically:
LD＝Es～p(s)[log D(s)]+Eb～p(b)[log(1-D(G(E(b))))]；
wherein L isDRepresenting a loss function of a discriminator, D representing the discriminator, G representing a generator, E representing an encoder, s representing a clear pollen particle picture with the highest feature similarity with the fuzzy pollen particle picture, b representing the fuzzy pollen particle picture, and p(s) and p (b) respectively representing the probability distribution of the clear pollen particle picture and the probability distribution of the fuzzy pollen particle picture.
5. The method for processing the blurred pollen particle picture as claimed in claim 2, wherein the calculating the feature similarity between the feature maps of the different classes of the distinct pollen particle pictures and the feature map of the blurred pollen particle picture specifically comprises:
wherein L issRepresenting a value of a characteristic difference, Φ, between the blurred and the clear pollen grain picturesi(s) feature map of the ith layer of the clear pollen grain picture,. phii(b) Feature map, M, representing the ith layer of a blurred pollen grain pictureiIndicating the size of the ith layer signature.
6. The method as claimed in claim 2, wherein before inputting the distinct pollen particle pictures of different categories into the encoder, the method further comprises:
cutting the different types of clear pollen particle pictures;
and randomly and horizontally rotating the clear pollen particle pictures of the categories with the number smaller than the preset value.
7. A fuzzy pollen particle picture processing device is characterized by comprising:
the input module is used for inputting the fuzzy pollen particle picture to be processed to generate a confrontation network model;
the output module is used for outputting a clear pollen particle picture;
the generated confrontation network model is obtained by carrying out unsupervised training by taking a fuzzy pollen particle picture and a clear pollen particle picture corresponding to the fuzzy pollen particle picture as sample data; the clear pollen particle pictures corresponding to the fuzzy pollen particle pictures are clear pollen particle pictures with the largest feature similarity with the fuzzy pollen particle pictures in different categories of clear pollen particle pictures.
8. The fuzzy pollen particle picture processing device as claimed in claim 7, wherein the clear pollen particle picture corresponding to the fuzzy pollen particle picture is a clear pollen particle picture with the largest feature similarity with the fuzzy pollen particle picture, among the clear pollen particle pictures with different categories, comprising:
inputting the different types of clear pollen particle pictures into an encoder, and outputting the feature maps of the different types of clear pollen particle pictures;
inputting the fuzzy pollen particle picture into an encoder, and outputting a characteristic diagram of the fuzzy pollen particle picture;
calculating the feature similarity between the feature maps of the different classes of clear pollen particle pictures and the feature map of the fuzzy pollen particle picture;
and determining the clear pollen particle picture corresponding to the feature picture of the clear pollen particle picture with the maximum feature similarity between the feature pictures of the fuzzy pollen particle picture as the clear pollen particle picture corresponding to the fuzzy pollen particle picture.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method for processing a blurred pollen grain picture according to any of claims 1 to 6 when executing the computer program.
10. A non-transitory computer readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the fuzzy pollen grain picture processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011255617.XA CN112435179A (en) | 2020-11-11 | 2020-11-11 | Fuzzy pollen particle picture processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011255617.XA CN112435179A (en) | 2020-11-11 | 2020-11-11 | Fuzzy pollen particle picture processing method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112435179A true CN112435179A (en) | 2021-03-02 |
Family
ID=74700866
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011255617.XA Pending CN112435179A (en) | 2020-11-11 | 2020-11-11 | Fuzzy pollen particle picture processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112435179A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019178893A1 (en) * | 2018-03-22 | 2019-09-26 | 深圳大学 | Motion blur image sharpening method and device, apparatus, and storage medium |
| CN111199522A (en) * | 2019-12-24 | 2020-05-26 | 重庆邮电大学 | Single-image blind motion blur removing method for generating countermeasure network based on multi-scale residual errors |
| CN111428575A (en) * | 2020-03-02 | 2020-07-17 | 武汉大学 | Tracking method for fuzzy target based on twin network |
| CN111738230A (en) * | 2020-08-05 | 2020-10-02 | 深圳市优必选科技股份有限公司 | Face recognition method, face recognition device and electronic equipment |
-
2020
- 2020-11-11 CN CN202011255617.XA patent/CN112435179A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019178893A1 (en) * | 2018-03-22 | 2019-09-26 | 深圳大学 | Motion blur image sharpening method and device, apparatus, and storage medium |
| CN111199522A (en) * | 2019-12-24 | 2020-05-26 | 重庆邮电大学 | Single-image blind motion blur removing method for generating countermeasure network based on multi-scale residual errors |
| CN111428575A (en) * | 2020-03-02 | 2020-07-17 | 武汉大学 | Tracking method for fuzzy target based on twin network |
| CN111738230A (en) * | 2020-08-05 | 2020-10-02 | 深圳市优必选科技股份有限公司 | Face recognition method, face recognition device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110378844B (en) | Image blind motion blur removing method based on cyclic multi-scale generation countermeasure network | |
| CN109003240B (en) | Image denoising method based on multi-scale parallel CNN | |
| CN111241958B (en) | Video image identification method based on residual error-capsule network | |
| CN113658051B (en) | Image defogging method and system based on cyclic generation countermeasure network | |
| CN111553406B (en) | Target detection system, method and terminal based on improved YOLO-V3 | |
| CN111696196B (en) | Three-dimensional face model reconstruction method and device | |
| CN111882643A (en) | Three-dimensional face construction method and device and electronic equipment | |
| CN107784288A (en) | A kind of iteration positioning formula method for detecting human face based on deep neural network | |
| CN111861894A (en) | Image motion blur removing method based on generating type countermeasure network | |
| CN110102051B (en) | Method and device for detecting game plug-in | |
| CN111369623B (en) | Lung CT image identification method based on deep learning 3D target detection | |
| CN116051408B (en) | Image depth denoising method based on residual error self-coding | |
| CN115601574A (en) | Unmanned aerial vehicle image matching method for improving AKAZE characteristics | |
| CN109948575A (en) | Eyeball dividing method in ultrasound image | |
| CN115131203A (en) | LR image generation method and real image super-resolution method based on uncertainty | |
| CN111488810A (en) | Face recognition method and device, terminal equipment and computer readable medium | |
| CN108305268A (en) | A kind of image partition method and device | |
| Jin et al. | Defect identification of adhesive structure based on DCGAN and YOLOv5 | |
| JP2021170284A (en) | Information processing device and program | |
| CN114283058A (en) | Image super-resolution reconstruction method based on countermeasure network and maximum mutual information optimization | |
| CN111476739B (en) | Underwater image enhancement method, system and storage medium | |
| CN109829857B (en) | Method and device for correcting inclined image based on generation countermeasure network | |
| CN115409950B (en) | Optimization method for surface drawing triangular mesh | |
| CN112435179A (en) | Fuzzy pollen particle picture processing method and device and electronic equipment | |
| CN116385281A (en) | Remote sensing image denoising method based on real noise model and generated countermeasure network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |