CN109902746A - Asymmetrical fine granularity IR image enhancement system and method - Google Patents
Asymmetrical fine granularity IR image enhancement system and method Download PDFInfo
- Publication number
- CN109902746A CN109902746A CN201910154431.6A CN201910154431A CN109902746A CN 109902746 A CN109902746 A CN 109902746A CN 201910154431 A CN201910154431 A CN 201910154431A CN 109902746 A CN109902746 A CN 109902746A
- Authority
- CN
- China
- Prior art keywords
- image
- infrared image
- fine granularity
- authentic specimen
- infrared
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 238000009826 distribution Methods 0.000 claims abstract description 22
- 238000012512 characterization method Methods 0.000 claims abstract description 19
- 238000005070 sampling Methods 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims description 19
- 230000004913 activation Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 5
- 230000008929 regeneration Effects 0.000 abstract 1
- 238000011069 regeneration method Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 26
- 230000008901 benefit Effects 0.000 description 5
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000003331 infrared imaging Methods 0.000 description 4
- 238000003786 synthesis reaction Methods 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000012827 research and development Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003475 lamination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The present invention provides a kind of asymmetrical fine granularity IR image enhancement system and methods, comprising: encoder is encoded for obtaining infrared image, and to the infrared image, obtains the characterization z of the authentic specimen x of infrared image；Generator, for obtaining the characterization z of authentic specimen, and by the generation sample x ' of infrared image being generated, by the authentic specimen and generations sample progress paired samples matching to distribution P (x | z, c) sampling；Arbiter, for the characteristics of mean of the authentic specimen to match with the characteristics of mean for generating sample；Classifier, for being fitted Posterior probability distribution P (c | x)；IR image enhancement model, for generating fine granularity condition image.The present invention can solve because generate confrontation model it is unstable due to make image failed regeneration the case where, while can with fine granularity control conditions of infrared images generation, the infrared image diversity authenticity of generation is preferable.
Description
Technical field
The present invention relates to image generating technologies field, in particular to a kind of asymmetrical fine granularity IR image enhancement system
And method.
Background technique
In recent years, it is always research hotspot that the image in deep learning field, which generates, and generates the most outstanding representative of model point
It is not to generate confrontation network G enerative Adversarial Networks (GAN) and variation automatic coding machine Variation
Auto-Encoder (VAE), VAE cause the image generated fuzzy since utilization mean square error constructs loss function, and traditional
GAN for how using characterization z there is no any restrictions, cause model training be difficult restrain and generate image it is often untrue.
Recently, many models attempt to improve the quality for generating sample.For example, WGAN uses target of the EM distance as training GAN,
MCGAN is matched using mean value and covariance feature.They need to limit the range of parameter, therefore discriminator can reduce discrimination.
Inventing a kind of image and generating model and can merge GAN and generate precision of images height and the stable advantage of VAE model training becomes research
It is popular.
Infrared imaging system has been widely used in optical remote sensing, night navigation, the military neck such as target acquisition and precise guidance
Domain has the features such as precision is high, strong antijamming capability using the weapon system of infrared imaging, these weapon systems are in R&D process
In, need mass data to be trained and assess, a large amount of true infrared images help to promote the performance of infrared imaging system,
But since the uncontrollable condition such as weather influences, all often taken a substantial amount of time using true infrared image and resource, together
When, these weapon systems itself are the complication systems for integrating multiple technologies, in order to shorten the R&D cycle, reduce research and development expense
With using IR image enhancement technology, effectively training and assessment infrared imaging system become a kind of effective means.
Summary of the invention
The present invention provides a kind of asymmetrical fine granularity IR image enhancement system and methods, and its purpose is to solve
IR image enhancement model training is unstable, and the image of generation is untrue, the few problem of type.
In order to achieve the above object, the embodiment provides a kind of asymmetrical fine granularity IR image enhancement systems
System, comprising:
Encoder is encoded for obtaining infrared image, and to the infrared image, obtains the true sample of infrared image
The characterization z of this x；
Generator, for obtaining the characterization z of authentic specimen, and by generating infrared image to distribution P (x | z, c) sampling
Generation sample x ', by the authentic specimen and the generations sample progress paired samples matching；
Arbiter, for the characteristics of mean of the authentic specimen to match with the characteristics of mean for generating sample；
Classifier, for being fitted Posterior probability distribution P (c | x)；
IR image enhancement model, for generating fine granularity condition image.
Wherein, the encoder is specifically used for converting a binary matrix, image tag for the infrared image of input
The one-hot vector for being converted to 10 dimensions carries out convolution to image array, and batch normalizes, and nonlinear activation operation is forced
The mean μ and covariance ε of near-infrared image authentic specimen distribution obtain the true of infrared image by formula z=μ+rexp (ε)
The characterization z of real sample.
Wherein, the arbiter by the characteristics of mean of the authentic specimen and generates the characteristics of mean of sample and matches needs
Loss function is minimized,
The formula of loss function is LD=-EX~Pr[logD(x)]-EZ~Pz[log(1-D(G(z))]；
The authentic specimen is matched with generation sample progress paired samples and is also required to make generator by the generator
Corresponding loss function minimizes,
The formula of loss function is
Wherein, fD(x) and fC(x) spy that true infrared image is input to arbiter and the full articulamentum of classifier is respectively indicated
Sign, fD(G (z)) and fD(x') indicate that the infrared image generated is input to the feature of the full articulamentum of arbiter, fC(x') it indicates to generate
Infrared image be input to the feature of the full articulamentum of classifier.
Wherein, the classifier is specifically used for taking x as inputting and exporting K dimensional vector, is then become using SOFTMAX function
At class probability；The output of each entry indicates posterior probability P (c | x), and in the training stage, classifier attempts to minimize SOFTMAX
Loss；Pass through formula:
LC=-EX~Pr[logP(c|x)]；
Wherein, the IR image enhancement model passes through formula L=LD+LC+λ1LG+λ2LGD+λ3LGCGenerate fine granularity condition
Image, and the IR image enhancement model selects the image x1 and x2 of a same category, is extracted using encoder potential
Vector Z1And Z2, a series of latent variable Z is obtained by linear interpolation,
The embodiments of the present invention also provide a kind of asymmetrical fine granularity IR image enhancement methods, comprising:
Step 1, infrared image is obtained, and the infrared image is encoded, obtains the authentic specimen x's of infrared image
Characterize z；
Step 2, the characterization z of authentic specimen is obtained, and by generating the generation of infrared image to distribution P (x | z, c) sampling
The authentic specimen and the generation sample are carried out paired samples matching by sample x '；
Step 3, the characteristics of mean of the authentic specimen is matched with the characteristics of mean for generating sample；
Step 4, it is fitted Posterior probability distribution P (c | x)；
Step 5, fine granularity condition image is generated.
Wherein, the step 2 specifically includes: converting a binary matrix, image tag for the infrared image of input
The one-hot vector for being converted to 10 dimensions carries out convolution to image array, and batch normalizes, and nonlinear activation operation is forced
The mean μ and covariance ε of near-infrared image authentic specimen distribution obtain the true of infrared image by formula z=μ+rexp (ε)
The characterization z of real sample.
Wherein, the step 2 and step 3 further comprise:
The formula of loss function is LD=-EX~Pr[logD(x)]-EZ~Pz[log(1-D(G(z))]；
The authentic specimen is matched with generation sample progress paired samples and is also required to make generator by the generator
Corresponding loss function minimizes,
The formula of loss function is
Wherein, fD(x) and fC(x) spy that true infrared image is input to arbiter and the full articulamentum of classifier is respectively indicated
Sign, fD(G (z)) and fD(x') indicate that the infrared image generated is input to the feature of the full articulamentum of arbiter, fC(x') it indicates to generate
Infrared image be input to the feature of the full articulamentum of classifier.
Wherein, the step 4 specifically includes: taking x as inputting and exporting K dimensional vector, is then become using SOFTMAX function
At class probability；The output of each entry indicates posterior probability P (c | x), and in the training stage, classifier attempts to minimize SOFTMAX
Loss；Pass through formula:
LC=-EX~Pr[logP(c|x)]；
Wherein, the step 5 specifically includes: passing through formula L=LD+LC+λ1LG+λ2LGD+λ3LGCGenerate fine granularity information drawing
Picture, and the IR image enhancement model select a same category image x1 and x2, using encoder extract it is potential to
Measure Z1And Z2, a series of latent variable Z is obtained by linear interpolation,
Above scheme of the invention have it is following the utility model has the advantages that
Asymmetrical fine granularity IR image enhancement system and method described in the above embodiment of the present invention uses mean value
Characteristic matching replaces the objective function of generator in traditional GAN, the objective function by synthesize sample feature center come
The center of the feature of actual sample is matched, the eigencenter for combining multiple network layers carrys out the convergence of acceleration model, enhances infrared life
At the stability of model；In order to generate different samples, increase an encoder network E to obtain from real image x to latent space z
Mapping, latent space vector z is inputed into generator G, synthesis sample x ' with authentic specimen x, be configured to feature (x, x ')；
Characteristics of mean is matched and is merged with the method for pairs of characteristic matching, the IR image enhancement model an of entirety is constructed.
Detailed description of the invention
Fig. 1 is the structural schematic diagram of asymmetrical fine granularity IR image enhancement system of the invention；
Fig. 2 is the flow chart of asymmetrical fine granularity IR image enhancement method of the invention；
Fig. 3 is infrared true picture figure of the invention；
Fig. 4 is infrared generation effect picture of the invention；
Fig. 5 is that infrared image attribute of the invention maps schematic diagram.
Specific embodiment
To keep the technical problem to be solved in the present invention, technical solution and advantage clearer, below in conjunction with attached drawing and tool
Body embodiment is described in detail.
The present invention is unstable for existing IR image enhancement model training, and the image of generation is untrue, and type is few
Problem provides a kind of asymmetrical fine granularity IR image enhancement system and method.
As shown in Figure 1, the embodiment provides a kind of asymmetrical fine granularity IR image enhancement system, packet
Include: encoder is encoded for obtaining infrared image, and to the infrared image, obtains the authentic specimen x's of infrared image
Characterize z；Generator, for obtaining the characterization z of authentic specimen, and by generating infrared image to distribution P (x | z, c) sampling
Sample x ' is generated, the authentic specimen and the generation sample are subjected to paired samples matching；Arbiter, being used for will be described true
The characteristics of mean of sample matches with the characteristics of mean for generating sample；Classifier, for being fitted Posterior probability distribution P (c | x)；It is red
Outer image generates model, for generating fine granularity condition image.
Asymmetrical fine granularity IR image enhancement system described in the above embodiment of the present invention uses characteristics of mean
Match to replace the objective function of generator in traditional GAN, which matches reality by the center of the feature of synthesis sample
The center of the feature of border sample, the eigencenter for combining multiple network layers carry out the convergence of acceleration model, enhance infrared generation model
Stability；In order to generate different samples, increase an encoder network E to obtain reflecting from real image x to latent space z
It penetrates, latent space vector z is inputed into generator G, synthesis sample x ' with authentic specimen x, be configured to feature (x, x ')；It will be equal
Value tag matching is merged with the method for pairs of characteristic matching, constructs the IR image enhancement model an of entirety.
Wherein, the encoder is specifically used for converting a binary matrix, image tag for the infrared image of input
The one-hot vector for being converted to 10 dimensions carries out convolution to image array, and batch normalizes, and nonlinear activation operation is forced
The mean μ and covariance ε of near-infrared image authentic specimen distribution obtain the true of infrared image by formula z=μ+rexp (ε)
The characterization z of real sample.
Encoder described in the above embodiment of the present invention encodes the infrared image of input, obtains infrared image
Characterize z, wherein global characteristics include the feature of entire image, and piece image is described as a row vector by it.Using
GoogleNet then carrys out training for promotion result from another angle as encoder, the proposition of inception: the more efficient benefit of energy
With computing resource, more features can be extracted under identical calculation amount, realize the mapping of true picture and latent space, thus
Study generates the structure of image.
Wherein, the arbiter by the characteristics of mean of the authentic specimen and generates the characteristics of mean of sample and matches needs
Loss function is minimized,
The formula of loss function is LD=-EX~Pr[logD(x)]-EZ~Pz[log(1-D(G(z))]；
The authentic specimen is matched with generation sample progress paired samples and is also required to make generator by the generator
Corresponding loss function minimizes,
The formula of loss function is
Wherein, fD(x) and fC(x) spy that true infrared image is input to arbiter and the full articulamentum of classifier is respectively indicated
Sign, fD(G (z)) and fD(x') indicate that the infrared image generated is input to the feature of the full articulamentum of arbiter, fC(x') it indicates to generate
Infrared image be input to the feature of the full articulamentum of classifier.
Wherein, the classifier is specifically used for taking x as inputting and exporting K dimensional vector, is then become using SOFTMAX function
At class probability；The output of each entry indicates posterior probability P (c | x), and in the training stage, classifier attempts to minimize SOFTMAX
Loss.Pass through formula:
LC=-EX~Pr[logP(c|x)]；
The average characteristics matching that the above embodiment of the present invention use condition image generates, it is assumed that we have one group to belong to K
The data of a classification, measure whether image belongs to specific fine granularity classification using network C, we are with standard here
Method is classified, and network C takes x as inputting and exporting K dimensional vector, then becomes class probability using softmax function.Each
The output of entry indicates posterior probability P (c | x).In the training stage, network C attempts to minimize softmax loss.
Generation network G described in the above embodiment of the present invention is by 2 layers of fully-connected network, 6 layers of deconvolution, adopts on 2*2
Sample sliding window, filter are designed and sized to 3*3,3*3,5*5,5*5,5*5,5*5；Using Alexnet as sorter network,
It is 128*128 by the infrared image size conversion of input, batch- is all added in each convolutional layer and warp lamination
Normalization, and 256 are set by infrared image characterization Z-dimension, image real information can be retained as far as possible in this way,
Entire model is constructed using deep learning frame pytorch, we select the last one FC layers of input of network C here, by
In in small batch only have minority belong to same category of sample, it is therefore necessary to use actual sample and generate sample movement
Average characteristics.Target is matched using the average characteristics of generator, which needs to synthesize the center of the feature of sample to match reality
The center of the feature of border sample.If fD(x) indicating the feature of arbiter middle layer, then generator attempts to keep loss function minimum,In an experiment, we have selected finally being fully connected for input network D
FC layers are used as feature.Convergence rate can be slightly improved in conjunction with multilayer feature.In the training stage, the number in mini-batch is used
Average characteristics according to estimates.
Therefore, in the training stage, we use LDAnd LGDNetwork D is updated, is trained GAN using this asymmetric loss
Have the advantages that following three: 1) l as separation property increases, on eigencenter2Loss solves the problems, such as gradient disappearance；2) work as life
At image it is good enough when, average characteristics match penalties become zero so that training it is more stable；3) compared with WGAN, we are not
Shear parameters are needed, the judgement index of network D can be kept.
It can prevent all outputs from shifting to single-point using average characteristics matching, thus a possibility that reducing mode collapse, but
Not fully solve this problem.Once mode collapse occurs, generating network is that different latent variables exports identical image,
Therefore general who has surrendered cannot separate these identical outputs under gradient.Moreover, the sample generated and the reality with same characteristic features center
Sample may have different distributions.
As shown in Figure 3 and Figure 4, for each sample, encoder network exports the mean value and covariance of latent variable, we
Latent variable z=μ+rexp (ε) can be sampled, wherein r~N (0, I) is a random vector, and " " indicates unit
Multiplication.It obtains after being mapped from x to z, we obtain the infrared image x' generated with network G is generated.Thus clearly reflected
The feature samples of relationship are penetrated to (x, x'), then, we add a l between x and x'2Rebuild loss and pairs of feature
With loss.
Wherein, the IR image enhancement model passes through formula L=LD+LC+λ1LG+λ2LGD+λ3LGCGenerate fine granularity condition
Image, and the IR image enhancement model selects the image x1 and x2 of a same category, is extracted using encoder potential
Vector Z1And Z2, a series of latent variable Z is obtained by linear interpolation,
λ is set1=1, λ2=10-3, λ3=10-3, each single item of above-mentioned formula is all meaningful, LPWith encoder network E
It is related with network D is differentiated.Whether it indicates the distribution of latent variable under expectation.LG, LGDAnd LGCIt is related with network G is generated.
Whether they respectively indicate composograph with input training sample, other samples in real image and same category are similar.
LGCRelated to sorter network C, sorter network C indicates the different classification of the ability that network classifies to image, LDWith differentiation net
Network is related, indicates that network is how well in terms of distinguishing true/composograph.All these targets are complementary to one another, and are finally made
Our algorithm obtains better result.
It attempts two kinds of data enhancing strategies: (1) generating more images for the existing classification that training data is concentrated, increase class
Between data；(2) new identity is generated by mixing different classifications, increases infrared data classification.In in this section, it is intended to test
The attribute that card generates in image will change with latent variable and constantly, this phenomenon is called attribute and mapped by we.Such as Fig. 5 institute
Show, our model is tested on infrared data collection, we select the image x1 and x2 of a same category first, then with volume
Code network E extracts latent variable Z1And Z2.Finally, obtaining a series of latent variable z by linear interpolation, i.e.,α ∈ [0,1] in each row, observe attribute such as posture, angle, color or illumination etc. from left to right by
Gradually situation of change.
The embodiments of the present invention also provide a kind of asymmetrical fine granularity IR image enhancement method, packets as shown in Figure 2
It includes:
Step 1, infrared image is obtained, and the infrared image is encoded, obtains the authentic specimen x's of infrared image
Characterize z；
Step 2, the characterization z of authentic specimen is obtained, and by generating the generation of infrared image to distribution P (x | z, c) sampling
The authentic specimen and the generation sample are carried out paired samples matching by sample x '；
Step 3, the characteristics of mean of the authentic specimen is matched with the characteristics of mean for generating sample；
Step 4, it is fitted Posterior probability distribution P (c | x)；
Step 5, fine granularity condition image is generated.
Wherein, the step 2 specifically includes: converting a binary matrix, image tag for the infrared image of input
The one-hot vector for being converted to 10 dimensions carries out convolution to image array, and batch normalizes, and nonlinear activation operation is forced
The mean μ and covariance ε of near-infrared image authentic specimen distribution obtain the true of infrared image by formula z=μ+rexp (ε)
The characterization z of real sample.
Wherein, the step 2 and step 3 further comprise:
The formula of loss function is LD=-EX~Pr[logD(x)]-EZ~Pz[log(1-D(G(z))]；
The authentic specimen is matched with generation sample progress paired samples and is also required to make generator by the generator
Corresponding loss function minimizes,
The formula of loss function is
Wherein, fD(x) and fC(x) spy that true infrared image is input to arbiter and the full articulamentum of classifier is respectively indicated
Sign, fD(G (z)) and fD(x') indicate that the infrared image generated is input to the feature of the full articulamentum of arbiter, fC(x') it indicates to generate
Infrared image be input to the feature of the full articulamentum of classifier.
Wherein, the step 4 specifically includes: taking x as inputting and exporting K dimensional vector, is then become using SOFTMAX function
At class probability；The output of each entry indicates posterior probability P (c | x), and in the training stage, classifier attempts to minimize SOFTMAX
Loss.Pass through formula:
LC=-EX~Pr[logP(c|x)]；
Wherein, the step 5 specifically includes: passing through formula L=LD+LC+λ1LG+λ2LGD+λ3LGCGenerate fine granularity information drawing
Picture, and the IR image enhancement model select a same category image x1 and x2, using encoder extract it is potential to
Measure Z1And Z2, a series of latent variable Z is obtained by linear interpolation,
Asymmetrical fine granularity IR image enhancement system and method described in the above embodiment of the present invention uses mean value
Characteristic matching replaces the objective function of generator in traditional GAN, the objective function by synthesize sample feature center come
The center of the feature of actual sample is matched, the eigencenter for combining multiple network layers carrys out the convergence of acceleration model, enhances infrared life
At the stability of model；In order to generate different samples, increase an encoder network E to obtain from real image x to latent space z
Mapping, latent space vector z is inputed into generator G, synthesis sample x ' with authentic specimen x, be configured to feature (x, x ')；
Characteristics of mean is matched and is merged with the method for pairs of characteristic matching, the IR image enhancement model an of entirety is constructed.
The above is a preferred embodiment of the present invention, it is noted that for those skilled in the art
For, without departing from the principles of the present invention, it can also make several improvements and retouch, these improvements and modifications
It should be regarded as protection scope of the present invention.
Claims (10)
1. a kind of asymmetrical fine granularity IR image enhancement system characterized by comprising
Encoder is encoded for obtaining infrared image, and to the infrared image, obtains the authentic specimen x of infrared image
Characterization z；
Generator, for obtaining the characterization z of authentic specimen, and by generating the life of infrared image to distribution P (x | z, c) sampling
At sample x ', the authentic specimen and the generation sample are subjected to paired samples matching；
Arbiter, for the characteristics of mean of the authentic specimen to match with the characteristics of mean for generating sample；
Classifier, for being fitted Posterior probability distribution P (c | x)；
IR image enhancement model, for generating fine granularity condition image.
2. asymmetrical fine granularity IR image enhancement system according to claim 1, which is characterized in that the encoder
Specifically for converting a binary matrix for the infrared image of input, image tag be converted to the one-hot of 10 dimensions to
Amount carries out convolution to image array, and batch normalizes, nonlinear activation operation, obtains approaching the distribution of infrared image authentic specimen
Mean μ and covariance ε by formula z=μ+rexp (ε), obtain the characterization z of the authentic specimen of infrared image.
3. asymmetrical fine granularity IR image enhancement system according to claim 2, which is characterized in that the arbiter
The characteristics of mean of the authentic specimen is matched with the characteristics of mean for generating sample and needs to minimize loss function,
The formula of loss function is LD=-EX~Pr[logD(x)]-EZ~Pz[log(1-D(G(z))]；
The authentic specimen is carried out paired samples matching with the generation sample and is also required to keep generator corresponding by the generator
Loss function minimize,
The formula of loss function is
Wherein, fD(x) and fC(x) feature that true infrared image is input to arbiter and the full articulamentum of classifier, f are respectively indicatedD
(G (z)) and fD(x') indicate that the infrared image generated is input to the feature of the full articulamentum of arbiter, fC(x') it indicates to generate red
Outer image is input to the feature of the full articulamentum of classifier.
4. asymmetrical fine granularity IR image enhancement system according to claim 3, which is characterized in that the classifier
Specifically for taking x as inputting and exporting K dimensional vector, then become class probability using SOFTMAX function；The output of each entry
Expression posterior probability P (c | x), in the training stage, classifier attempts to minimize the loss of SOFTMAX；Pass through formula:
LC=-EX~Pr[logP(c|x)]；
5. asymmetrical fine granularity IR image enhancement system according to claim 4, which is characterized in that the infrared figure
Pass through formula L=L as generating modelD+LC+λ1LG+λ2LGD+λ3LGCGenerate fine granularity condition image, and the IR image enhancement
Model selects the image x1 and x2 of a same category, extracts latent variable Z using encoder1And Z2, obtained by linear interpolation
To a series of latent variable Z,
6. a kind of asymmetrical fine granularity IR image enhancement method characterized by comprising
Step 1, infrared image is obtained, and the infrared image is encoded, obtains the characterization of the authentic specimen x of infrared image
z；
Step 2, the characterization z of authentic specimen is obtained, and by generating the generation sample of infrared image to distribution P (x | z, c) sampling
The authentic specimen and the generation sample are carried out paired samples matching by x '；
Step 3, the characteristics of mean of the authentic specimen is matched with the characteristics of mean for generating sample；
Step 4, it is fitted Posterior probability distribution P (c | x)；
Step 5, fine granularity condition image is generated.
7. asymmetrical fine granularity IR image enhancement method according to claim 6, which is characterized in that the step 2
Specifically include: converting a binary matrix for the infrared image of input, image tag be converted to the one-hot of 10 dimensions to
Amount carries out convolution to image array, and batch normalizes, nonlinear activation operation, obtains approaching the distribution of infrared image authentic specimen
Mean μ and covariance ε by formula z=μ+rexp (ε), obtain the characterization z of the authentic specimen of infrared image.
8. asymmetrical fine granularity IR image enhancement method according to claim 7, which is characterized in that the step 2
Further comprise with step 3:
The formula of loss function is LD=-EX~Pr[logD(x)]-EZ~Pz[log(1-D(G(z))]；
The authentic specimen is carried out paired samples matching with the generation sample and is also required to keep generator corresponding by the generator
Loss function minimize,
The formula of loss function is
Wherein, fD(x) and fC(x) feature that true infrared image is input to arbiter and the full articulamentum of classifier, f are respectively indicatedD
(G (z)) and fD(x') indicate that the infrared image generated is input to the feature of the full articulamentum of arbiter, fC(x') it indicates to generate red
Outer image is input to the feature of the full articulamentum of classifier.
9. asymmetrical fine granularity IR image enhancement method according to claim 8, which is characterized in that the step 4
It specifically includes: taking x as inputting and exporting K dimensional vector, then become class probability using SOFTMAX function；Each entry it is defeated
It indicates out posterior probability P (c | x), in the training stage, classifier attempts to minimize the loss of SOFTMAX；Pass through formula:
LC=-EX~Pr[logP(c|x)]；
10. asymmetrical fine granularity IR image enhancement method according to claim 8, which is characterized in that the step 5
It specifically includes: by formula L=LD+LC+λ1LG+λ2LGD+λ3LGCGenerate fine granularity condition image, and the IR image enhancement
Model selects the image x1 and x2 of a same category, extracts latent variable Z using encoder1And Z2, obtained by linear interpolation
To a series of latent variable Z,
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910154431.6A CN109902746A (en) | 2019-03-01 | 2019-03-01 | Asymmetrical fine granularity IR image enhancement system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910154431.6A CN109902746A (en) | 2019-03-01 | 2019-03-01 | Asymmetrical fine granularity IR image enhancement system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109902746A true CN109902746A (en) | 2019-06-18 |
Family
ID=66946074
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910154431.6A Pending CN109902746A (en) | 2019-03-01 | 2019-03-01 | Asymmetrical fine granularity IR image enhancement system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109902746A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI715427B (en) * | 2019-07-16 | 2021-01-01 | 大陸商北京市商湯科技開發有限公司 | Image processing method, processor, electronic equipment and computer readable storage medium thereof |
| CN113052203A (en) * | 2021-02-09 | 2021-06-29 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Anomaly detection method and device for multiple types of data |
| CN115994966A (en) * | 2023-03-22 | 2023-04-21 | 北京红棉小冰科技有限公司 | Multi-view image generation method and device, readable storage medium and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108875818A (en) * | 2018-06-06 | 2018-11-23 | 西安交通大学 | Based on variation from code machine and confrontation network integration zero sample image classification method |
| CN109146988A (en) * | 2018-06-27 | 2019-01-04 | 南京邮电大学 | Non-fully projection CT image rebuilding method based on VAEGAN |
-
2019
- 2019-03-01 CN CN201910154431.6A patent/CN109902746A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108875818A (en) * | 2018-06-06 | 2018-11-23 | 西安交通大学 | Based on variation from code machine and confrontation network integration zero sample image classification method |
| CN109146988A (en) * | 2018-06-27 | 2019-01-04 | 南京邮电大学 | Non-fully projection CT image rebuilding method based on VAEGAN |
Non-Patent Citations (6)
| Title |
|---|
| J. MA ET AL.: "FusionGAN: A generative adversarial network for infrared and visible image fusion", 《INFORMATION FUSION》 * |
| JIANMIN BAO，ET AL.: "CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training", 《ARXIV》 * |
| MAHESH GORIJALA， ET AL.: "Image Generation and Editing with Variational Info Generative Adversarial Networks", 《ARXIV》 * |
| 王雅欣，ET AL.: "基于生成对抗网络的人脸热红外图像生成", 《信息技术与网络安全》 * |
| 辛阳，等.: "《大数据技术原理与实践》", 31 March 2018, 北京邮电大学出版社 * |
| 高志强，等.: "《深度学习从入门到实战》", 30 June 2018, 中国铁道出版社 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI715427B (en) * | 2019-07-16 | 2021-01-01 | 大陸商北京市商湯科技開發有限公司 | Image processing method, processor, electronic equipment and computer readable storage medium thereof |
| CN113052203A (en) * | 2021-02-09 | 2021-06-29 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Anomaly detection method and device for multiple types of data |
| CN113052203B (en) * | 2021-02-09 | 2022-01-18 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Anomaly detection method and device for multiple types of data |
| CN115994966A (en) * | 2023-03-22 | 2023-04-21 | 北京红棉小冰科技有限公司 | Multi-view image generation method and device, readable storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jia et al. | A semisupervised Siamese network for hyperspectral image classification | |
| CN111476294B (en) | Zero sample image identification method and system based on generation countermeasure network | |
| CN112232156B (en) | Remote sensing scene classification method based on multi-head attention generation countermeasure network | |
| CN103473786B (en) | Gray level image segmentation method based on multi-objective fuzzy clustering | |
| Huang et al. | Alleviating modality bias training for infrared-visible person re-identification | |
| Wu et al. | Ordered subspace clustering with block-diagonal priors | |
| CN109902746A (en) | Asymmetrical fine granularity IR image enhancement system and method | |
| CN113239801B (en) | Cross-domain action recognition method based on multi-scale feature learning and multi-level domain alignment | |
| CN114386534A (en) | Image augmentation model training method and image classification method based on variational self-encoder and countermeasure generation network | |
| Zhou et al. | SAR target recognition via joint sparse representation of monogenic components with 2D canonical correlation analysis | |
| CN109978042A (en) | A kind of adaptive quick K-means clustering method of fusion feature study | |
| CN109376787A (en) | Manifold learning network and computer visual image collection classification method based on it | |
| Liang et al. | Spectral–spatial attention feature extraction for hyperspectral image classification based on generative adversarial network | |
| Peng et al. | Convolutional transformer-based few-shot learning for cross-domain hyperspectral image classification | |
| Ye et al. | Adjacent-level feature cross-fusion with 3D CNN for remote sensing image change detection | |
| Lu et al. | Dynamic ensemble learning with multi-view kernel collaborative subspace clustering for hyperspectral image classification | |
| Pradhyumna | A survey of modern deep learning based generative adversarial networks (gans) | |
| Li et al. | Mixing Self-Attention and Convolution: A Unified Framework for Multi-source Remote Sensing Data Classification | |
| CN103942572A (en) | Method and device for extracting facial expression features based on bidirectional compressed data space dimension reduction | |
| CN113052017A (en) | Unsupervised pedestrian re-identification method based on multi-granularity feature representation and domain adaptive learning | |
| CN116704612A (en) | Cross-visual-angle gait recognition method based on contrast domain self-adaptive learning | |
| Zheng et al. | Hybrid FusionNet: A Hybrid Feature Fusion Framework for Multi-Source High-Resolution Remote Sensing Image Classification | |
| Chi et al. | Cluster-based ensemble classification for hyperspectral remote sensing images | |
| Cheng et al. | Unified Classification and Rejection: A One-versus-All Framework | |
| CN113128471A (en) | Hyperspectral image classification method based on attention mechanism and generation countermeasure network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190618 |