CN115439573B - Method for generating visible light-to-infrared image based on temperature information coding - Google Patents
Method for generating visible light-to-infrared image based on temperature information coding Download PDFInfo
- Publication number
- CN115439573B CN115439573B CN202211324117.6A CN202211324117A CN115439573B CN 115439573 B CN115439573 B CN 115439573B CN 202211324117 A CN202211324117 A CN 202211324117A CN 115439573 B CN115439573 B CN 115439573B
- Authority
- CN
- China
- Prior art keywords
- temperature
- image
- infrared
- visible light
- self
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000010438 heat treatment Methods 0.000 claims abstract description 64
- 238000012549 training Methods 0.000 claims abstract description 27
- 238000009432 framing Methods 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 12
- 238000004088 simulation Methods 0.000 claims description 11
- 238000012952 Resampling Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 abstract 1
- 230000002269 spontaneous effect Effects 0.000 abstract 1
- 230000003287 optical effect Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000003331 infrared imaging Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/001—Texturing; Colouring; Generation of texture or colour
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/48—Thermography; Techniques using wholly visual means
- G01J5/485—Temperature profile
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/32—Normalisation of the pattern dimensions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Abstract
The invention discloses a method for generating a visible light-infrared image based on temperature information coding, which comprises the following steps: acquiring many-to-one paired original visible light images and original infrared images, framing a spontaneous heating target in the images, and preprocessing to obtain a training set; training a pre-constructed generation model considering temperature information by using a training set until the model converges; in each round of training process, encoding the global temperature and the self-heating target temperature of the original infrared image set to obtain the temperature distribution of the current image pair; and inputting the visible light image to be converted and the temperature code sampled from the temperature distribution into the trained generation model to obtain a plurality of simulated infrared images under different temperature states. The invention considers the inconsistency of the temperature needed by the target and the background, the temperature of the target can self-heat, the background is relatively stable, and the infrared images with different temperature states are finally generated by combining different sampled temperature codes in the temperature distribution according to the content of the visible light.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a method for generating a visible light-to-infrared image based on temperature information coding.
Background
In recent years, the deep learning technology has been rapidly developed, and the excellent performance of the deep neural network is greatly dependent on massive data. However, in practice, the number of infrared images of many scenes is small, so that training of a model is difficult to support, and more data must be acquired to ensure that a task is completed well. With the continuous progress of the infrared imaging technology, rapid development is achieved, and currently, an intelligent detection system for infrared needs to use an intelligent sensing model to accurately and efficiently identify and position a target in upgrading and modifying, and the target can be accurately captured in various natural and man-made interference environments, so that a large number of infrared training samples under different imaging conditions and different interference scenes are needed, and the system is tested and evaluated in large scale under different scenes, so that the robustness of the system is ensured.
Because the infrared acquisition equipment is expensive, especially in military application, the cost of using a target drone or a missile for experiments is very high, a large amount of manpower, material resources and financial resources are consumed, and due to the limitation of test conditions such as climate, scenes and the like, only infrared images of specific meteorological conditions and specific scenes can be obtained. It is difficult to evaluate the actual combat performance of the weapon system under different state conditions. Therefore, the method for acquiring the infrared image of the algorithm test through simulation is a common mode, the infrared imaging simulation technology can provide an extremely effective and economic way for solving the problems, and the method has very important significance for developing and researching modern accurate guided weapons. Aiming at the problem, data enhancement and infrared simulation methods are often adopted to expand infrared samples, the data enhancement needs to be based on the existing infrared samples, the content of the images cannot be changed, the improvement on training is limited, the sample expansion based on the infrared simulation is a new trend at present, and some mature methods can generate the simulated infrared samples in a large batch.
However, existing infrared generation techniques have some limitations, which currently fall into two main categories: modeling and simulating an infrared scene and converting a visible light image into an infrared image. The simulation model based on the infrared scene modeling has strong interpretability, but needs a large amount of calculation and is slow in generation speed. Some infrared simulation algorithms based on visible light images are time-saving and labor-saving, but lack corresponding physical theory support, and are difficult to verify the infrared physical characteristics of the generated results. For the visible light converted infrared image, the generation method based on the deep learning does not consider the basic principle, and the simple fitting mode does not consider the basic principle of the physical layer, so that the generated sample cannot reflect the radiation characteristic of the thermal infrared, and the generation result of the existing generation method based on the deep learning lacks diversity.
In the background of current research, in order to deal with the problem that a visible light image can only generate a single infrared image and the problem that a certain physical degree of interpretation is lacked, a method for generating a visible light-to-infrared image based on temperature information coding is designed, which is a problem that needs to be solved urgently by technical personnel in the field.
Disclosure of Invention
In view of the above, the present invention provides a method for generating a visible light-to-infrared image based on temperature information coding, which considers that the temperatures required by a target and a background are not the same, the temperature of the target is self-heating, the background is relatively stable, temperature codes sampled in different ways from temperature distribution are combined according to the contents of different visible lights, and finally, infrared images with different temperature states are generated through a generation network.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for generating a visible light-to-infrared image based on temperature information coding comprises the following steps:
acquiring many-to-one paired original visible light images and original infrared images, unifying the image sizes, framing the self-heating targets in the images, and resampling the self-heating targets framed to be consistent with the original image sizes to obtain self-heating target images;
forming a sample group by the original visible light image and the self-heating target image thereof, the original infrared image and the self-heating target image thereof in each pair, and forming a training set by all the sample groups;
training a pre-constructed generation model considering temperature information by using a training set until the model converges; the generation model considering the temperature information comprises a temperature encoder, an infrared generator and a discriminator; in each round of training process, the temperature encoder encodes the global temperature and the self-heating target temperature of the original infrared image, and enables the temperature encoding of all data to be in accordance with the sampling from normal distribution, so as to obtain the temperature distribution of the current paired image; combining the content of the original visible light image and the temperature code sampled from the temperature distribution by the infrared generator to generate a simulated infrared image; the discriminator judges whether the infrared image is a simulated infrared image or a real original infrared image so as to guide the generation of the infrared generator to be more true;
and inputting the visible light image to be converted and different temperature codes obtained by sampling from the temperature distribution into the trained infrared generator to obtain a plurality of simulated infrared images under different temperature states.
Further, the process of forming the training set includes:
acquiring visible light image set
And a corresponding infrared image set->
Wherein, W × H represents the size of the visible light image and the infrared image, and N represents the number of the visible light image and the infrared image; "3" and "1" denote the number of channels, X, of the visible image and the infrared image, respectively i Representing the ith visible image, Y i Representing the ith infrared image;
framing self-heating targets in visible and infrared images, X i And Y i The object set contained is defined as
Represents Z i The jth target of (1); />
Representing the visible light images to be converted, and M representing the number of the visible light images to be converted;
after each self-heating target selected by each frame is deducted from the visible light image and the infrared image respectively, the self-heating target image of the visible light image is obtained by resampling to be consistent with the original image in size
Self-heating target image->
Further, the forward propagation formula of the temperature encoder and the infrared generator in the training stage is as follows:
wherein G (-) represents an infrared generator, E (-) represents a temperature encoder; n is a radical of i Represents X i And Y i The corresponding global temperature code is encoded with a global temperature,
represents->
And &>
Encoding the self-heating target temperature; suppose N i And &>
Respectively sampled according to independent multi-normal distribution>
Represents X i And Y i The self-heating target temperature coding set; />
Represents the generated simulated infrared image->
Represents the generated simulated infrared self-heating target image, and>
represents a global temperature coding, < '> or <' > of the generated simulated infrared image>
Representing the temperature encoding of the generated simulated infrared target image.
Further, the loss function of the generative model considering the temperature information is as follows:
wherein D (-) represents a discriminator, and L1, L2, L5 and L6 losses use an L1 loss function; l1 and l2 represent the mapping relation of converting a visible light image into an infrared image and capturing low-frequency information in the image, wherein the mapping relation is expected to be effectively learned; l5 and l6 are used for ensuring that the temperature code of the real infrared image is aligned with the code of the corresponding simulated infrared image, and ensuring that the infrared generator and the temperature encoder function simultaneously; l3 and l4 represent that the discriminator is used for judging whether the image is a real infrared image/self-heating target image or a generated simulation infrared image/self-heating target image, and the generated infrared image domain is guided to be close to the real infrared image domain as far as possible; i (-) indicates that if the real infrared image is 1, the simulation infrared image is 0; l7 is a KL loss function, which shows that the temperature code distribution of the real infrared images is in a normal distribution as much as possible,
representing a multivariate normal distribution, with dimensions consistent with the temperature code, u representing the set of mean values, σ 2 Representing a variance group; />
Representing a multivariate standard normal distribution.
Further, inputting the visible light image to be converted and different temperature codes obtained by sampling from the temperature distribution into the trained infrared generator to obtain a plurality of simulated infrared images under different temperature states, including:
taking out the infrared generator from the trained generation model considering temperature information, and acquiring the visible light image to be converted
And the self-heating target selected by the corresponding frame;
for a visible light image to be converted
From the temperature distribution->
Sampling global temperature coding N according to content of visible light image to be converted i And a self-heating target temperature code->
Give/pick>
Giving different global temperature codes and self-heating target temperature codes; the temperature coding can be changed, specifically, the temperature coding is directly superposed to the channel direction of the visible light image, the same temperature coding is added to each pixel of the global background information, the dimension of the temperature coding becomes a new channel, and the temperature coding of each target is added to all pixel positions in the same way.
And inputting the optical image to be converted, the global temperature code corresponding to the optical image and the temperature codes of different self-heating targets into an infrared generator together, and outputting a plurality of simulated infrared images in different temperature states.
Further, the conversion formula of the visible light image to be converted is as follows:
in the above formula, different temperature codes are sampled to the whole situation and the target of the visible light image to be converted, and a plurality of simulated infrared images in different temperature states are output.
According to the technical scheme, compared with the prior art, the method for generating the visible light-to-infrared image based on the temperature information coding is disclosed, and the existing visible light image and infrared image are utilized, and the designed infrared generator and temperature coder are used for respectively learning the process of converting the visible light into the infrared image and the distribution state of the temperature information. In a trained model, when a large number of visible light images are used for conversion, the temperature needed by a target and a background is considered to be inconsistent, the temperature of the target can be self-heated, the background is relatively stable, temperature codes sampled in different modes from temperature distribution are combined according to the content of different visible light, the generation of visible light to infrared images is performed by combining relative temperature information, the relative temperature without knowing a real temperature value can be coded into normal distribution, the relative temperature information is obtained by sampling, and finally infrared images with different temperature states are generated through an infrared generator.
Meanwhile, the same image can be respectively generated with infrared global information of different temperature codes, the same target can be replaced with different proper temperature codes to generate targets with different temperature combinations under the same background, and the generation scheme can generate infrared images in more various temperature states.
The invention also utilizes the strong fitting ability of the neural network to learn the accurate texture mapping and temperature coding from the visible light image to the infrared image, and the infrared image generated by the method can be used for other tasks, such as expanding the training data set of the infrared target detector.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a method for generating a visible light-to-infrared image based on temperature information encoding according to the present invention;
FIG. 2 is a flowchart illustrating infrared conversion of a visible light image to be converted by the trained infrared generator according to the present invention;
fig. 3 (a) is a temperature code of a fixed self-heating target provided by the invention, and different global false color images are converted.
Fig. 3 (b) is a fixed global coding provided by the present invention, and different self-heating target temperature codes are sampled to form different infrared images.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1-2, the embodiment of the invention discloses a method for generating a visible light-to-infrared image based on temperature information coding, which comprises the following steps:
s1, acquiring many-to-one paired original visible light images and original infrared images, unifying the image sizes, framing a self-heating target in the images, and resampling the self-heating target framed to be consistent with the original image size to obtain a self-heating target image;
forming a sample group by the original visible light image and the self-heating target image thereof, the original infrared image and the self-heating target image thereof in each pair, and forming a training set by all the sample groups;
s2, training a pre-constructed generation model considering temperature information by using a training set until the model converges; the generation model considering the temperature information comprises a temperature encoder, an infrared generator and a discriminator; in each round of training process, the temperature encoder encodes the global temperature and the self-heating target temperature of the original infrared image, and enables the temperature encoding of all data to be in accordance with the sampling from normal distribution, so as to obtain the temperature distribution of the current paired image; the infrared generator combines the content of the original visible light image and the temperature code sampled from the temperature distribution to generate a simulated infrared image; the discriminator judges whether the infrared image is a simulated infrared image or a real original infrared image so as to guide the generation of the infrared generator to be more true;
and S3, inputting the visible light image to be converted and different temperature codes obtained by sampling from the temperature distribution into the trained infrared generator to obtain a plurality of simulated infrared images under different temperature states.
In the embodiment of the invention, the temperature encoder can encode the global temperature information of the infrared image and the temperature information of the target in the training process, and can learn the temperature distribution state; sending the content of the visible light image and the sampled temperature code into an infrared generator to generate a simulated infrared image; the generated infrared and the real infrared are then sent to a discriminator for discrimination in order to generate a more similar.
In one embodiment, in S1, the process of forming the training set includes:
s11, acquiring a visible light image set
And a corresponding set of infrared images Y ∈ @>
(ii) a Wherein, W × H represents the size of the visible light image and the infrared image, and N represents the number of the visible light image and the infrared image; "3" and "1" denote the number of channels, X, of the visible image and the infrared image, respectively i Representing the ith visible image, Y i Representing the ith infrared image;
s12, framing the self-heating target in the visible light image and the infrared image, X i And Y i The object set contained is defined as
Represents Z i The jth target of (1); />
Representing the visible light images to be converted, and M representing the number of the visible light images to be converted;
s13, after the self-heating target selected by each frame is deducted from the visible light image and the infrared image respectively, resampling the self-heating target to be consistent with the original image in size, and obtaining a self-heating target image of the visible light image
Self-heating target image of infrared image->
In one embodiment, in S2, the forward propagation equations of the model training phase, the temperature encoder and the infrared generator are as follows:
wherein G (-) represents an infrared generator, E (-) represents a temperature encoder; n is a radical of i Represents X i And Y i The corresponding global temperature code is encoded with a global temperature,
represents->
And &>
Encoding the self-heating target temperature; suppose N i And &>
Respectively sampled according to independent multi-normal distribution>
Represents X i And Y i The self-heating target temperature coding set; />
Represents the generated simulated infrared image->
Represents the generated simulated infrared self-heating target image, and>
represents a global temperature coding, < '> or <' > of the generated simulated infrared image>
Representing the temperature encoding of the generated simulated infrared target image.
The loss function of the generative model taking into account the temperature information is as follows:
wherein D (-) represents a discriminator, and L1, L2, L5 and L6 losses use an L1 loss function; l1 and l2 represent the mapping relation of converting a visible light image into an infrared image and capturing low-frequency information in the image, wherein the mapping relation is expected to be effectively learned; l5 and l6 are used for ensuring that the temperature code of the real infrared image is aligned with the code of the corresponding simulated infrared image, and ensuring that the infrared generator and the temperature encoder function simultaneously; l3 and l4 represent that the discriminator is used for judging whether the image is a real infrared image/self-heating target image or a generated simulation infrared image/self-heating target image, and the generated infrared image domain is guided to be close to the real infrared image domain as far as possible; i (-) indicates that if the real infrared image is 1, the simulation infrared image is 0; l7 is the KL loss function, indicating that these real infrared images are as good as possibleIs distributed in a normal distribution,
representing a multivariate normal distribution, with dimensions consistent with the temperature code, u representing the set of mean values, σ 2 Representing a variance group; />
Representing a multivariate standard normal distribution.
In one embodiment, as shown in fig. 2, S3 comprises:
s31, taking out the infrared generator from the trained generation model considering the temperature information, and acquiring the visible light image to be converted
And the self-heating target selected by the corresponding frame;
s32, aiming at a visible light image to be converted
From the temperature distribution->
Sampling corresponding global temperature code N according to content of visible light image to be converted i And a self-heating target temperature code->
Give/pick>
Giving different global temperature codes and self-heating target temperature codes; the temperature coding can be changed, specifically, the temperature coding is directly superposed to the channel direction of the visible light image, the same temperature coding is added to each pixel of the global background information, the dimension of the temperature coding becomes a new channel, and the temperature coding of each target is added to all pixel positions in the same way.
S33, inputting the optical image to be converted, the global temperature code corresponding to the optical image and the temperature codes of different self-heating targets into an infrared generator together, and outputting a plurality of simulated infrared images corresponding to the sampled temperature codes in different temperature states; the conversion formula is as follows:
different temperature codes can be sampled to the global and target of the visible light to be converted, the temperature coding information added to the global and target can be changed to a certain extent, so that infrared images of various temperature states can be generated by using one visible light image, and the difference is caused by the temperature codes, as shown in fig. 3 (a) -3 (b).
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed in the embodiment corresponds to the method disclosed in the embodiment, so that the description is simple, and the relevant points can be referred to the description of the method part.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (3)
1. A method for generating a visible light-to-infrared image based on temperature information coding is characterized by comprising the following steps:
acquiring many-to-one paired original visible light images and original infrared images, unifying the image sizes, framing the self-heating targets in the images, and resampling the self-heating targets framed to be consistent with the original image sizes to obtain self-heating target images;
forming a sample group by the original visible light image and the self-heating target image thereof, the original infrared image and the self-heating target image thereof in each pair, and forming a training set by all the sample groups;
training a pre-constructed generation model considering temperature information by using a training set until the model converges; the generation model considering the temperature information comprises a temperature encoder, an infrared generator and a discriminator; in each round of training process, the temperature encoder encodes the global temperature and the self-heating target temperature of the original infrared image, and enables the temperature codes of all data to accord with normal distribution sampling to obtain the temperature distribution of the current paired image; the infrared generator combines the content of the original visible light image and the temperature code sampled from the temperature distribution to generate a simulated infrared image; sending the generated simulated infrared image and the real infrared image to a discriminator for judgment so as to guide the generation of an infrared generator;
inputting a visible light image to be converted and different temperature codes obtained by sampling from temperature distribution into a trained infrared generator to obtain a plurality of simulated infrared images in different temperature states;
the process of forming the training set comprises the following steps:
acquiring visible light image set
And corresponding infrared image set
Wherein, W × H represents the size of the visible light image and the infrared image, and N represents the number of the visible light image and the infrared image; "3" and "1" denote the number of channels, X, of the visible image and the infrared image, respectively i Representing the ith visible image, Y i Representing the ith infrared image;
framing self-heating targets in visible and infrared images, X i And Y i The object set contained is defined as
Represents Z i The jth target of (1);
representing the visible light images to be converted, and M representing the number of the visible light images to be converted;
after each self-heating target selected by each frame is deducted from the visible light image and the infrared image respectively, the self-heating target image of the visible light image is obtained by resampling to be consistent with the original image in size
Self-heating target image of infrared image
The forward propagation equations for the temperature encoder and infrared generator during the training phase are as follows:
wherein G (-) represents an infrared generator, E (-) represents a temperature encoder; n is a radical of i Represents X i And Y i The corresponding global temperature code is encoded with a global temperature,
represent
And
self-heating target temperature coding; suppose N i And
are respectively and independently subjected to multivariate normal distribution
Multivariate normal distribution
The dimensions are consistent with the temperature coding, u represents the set of mean values, σ 2 Representing a variance group;
represents X i And Y i The self-heating target temperature coding set;
representing the generated simulated infrared image,
representing the generated simulated infrared self-heating target image,
a global temperature encoding representing the generated simulated infrared image,
representing a temperature encoding of the generated simulated infrared target image;
the loss function of the generative model taking into account the temperature information is as follows:
wherein D (-) represents a discriminator, and l1, l2, l5 and l6 losses use a T1 loss function; l1 and l2 are used for learning the mapping relation of converting the visible light image into the infrared image and capturing low-frequency information in the image; l5 and l6 are used for ensuring that the temperature code of the real infrared image is aligned with the code of the corresponding simulation infrared image, and ensuring that the infrared generator and the temperature encoder function simultaneously; l3 represents a real infrared image or a generated simulated infrared image judged by the discriminator, l4 represents a self-heating target image of the real infrared image or the generated simulated infrared image judged by the discriminator, and l3 and l4 are used for guiding the generated infrared image domain to be close to the real infrared image domain; i (-) indicates that if the real infrared image is 1, the simulation infrared image is 0; l7 is a KL loss function which indicates that the temperature coding distribution of the real infrared image tends to be in normal distribution;
representing a multivariate standard normal distribution.
2. The method for generating the visible light-to-infrared image based on the temperature information code according to claim 1, wherein the visible light image to be converted and the different temperature codes obtained by sampling the temperature distribution are input into a trained infrared generator to obtain a plurality of simulated infrared images under different temperature states, and the method comprises:
taking out the infrared generator from the trained generation model considering temperature information, and acquiring a visible light image to be converted and a self-heating target selected by a frame corresponding to the visible light image;
for the ith visible light image to be converted
From the temperature distribution
According to the content of the visible light image to be converted, sampling the corresponding global temperature code N i And self-heating target temperature coding set
To give
Different global temperature codes and self-heating target temperature codes are given, specifically, the temperature codes are directly superposed to the channel direction of the visible light image, each pixel of global background information is added with the same temperature code, the dimension of the global background information becomes a new channel, and all pixel positions of each self-heating target are added with the temperature codes thereof in the same way;
and inputting the visible light image to be converted, the global temperature code corresponding to the visible light image and the temperature codes of different self-heating targets into an infrared generator together, and outputting a plurality of simulated infrared images in different temperature states.
3. The method for generating the visible-light-to-infrared image based on the temperature information coding as claimed in claim 2, wherein the conversion formula of the visible-light image to be converted is as follows:
in the above formula, different temperature codes are sampled to global and self-heating targets of the visible light image to be converted, and a plurality of simulated infrared images in different temperature states are output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211324117.6A CN115439573B (en) | 2022-10-27 | 2022-10-27 | Method for generating visible light-to-infrared image based on temperature information coding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211324117.6A CN115439573B (en) | 2022-10-27 | 2022-10-27 | Method for generating visible light-to-infrared image based on temperature information coding |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115439573A CN115439573A (en) | 2022-12-06 |
| CN115439573B true CN115439573B (en) | 2023-03-24 |
Family
ID=84252411
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211324117.6A Active CN115439573B (en) | 2022-10-27 | 2022-10-27 | Method for generating visible light-to-infrared image based on temperature information coding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115439573B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112699912A (en) * | 2020-11-19 | 2021-04-23 | 电子科技大学 | Method for enhancing infrared thermal image by improving GAN |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7299292B2 (en) * | 2016-04-19 | 2023-06-27 | マクセル株式会社 | Work support device |
| CN111145131B (en) * | 2019-11-28 | 2023-05-26 | 中国矿业大学 | Infrared and visible light image fusion method based on multiscale generation type countermeasure network |
| CN112163988B (en) * | 2020-08-17 | 2022-12-13 | 中国人民解放军93114部队 | Infrared image generation method and device, computer equipment and readable storage medium |
-
2022
- 2022-10-27 CN CN202211324117.6A patent/CN115439573B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112699912A (en) * | 2020-11-19 | 2021-04-23 | 电子科技大学 | Method for enhancing infrared thermal image by improving GAN |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115439573A (en) | 2022-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108846418B (en) | Cable equipment temperature abnormity positioning and identifying method | |
| CN105719188A (en) | Method and server for achieving insurance claim anti-fraud based on consistency of multiple pictures | |
| CN110705406A (en) | Face beauty prediction method and device based on transfer learning resistance | |
| CN114758252B (en) | Image-based distributed photovoltaic roof resource segmentation and extraction method and system | |
| CN109919921B (en) | Environmental impact degree modeling method based on generation countermeasure network | |
| CN106951863B (en) | Method for detecting change of infrared image of substation equipment based on random forest | |
| CN115561243B (en) | Pole piece quality monitoring system and method in lithium battery preparation | |
| CN113255590A (en) | Defect detection model training method, defect detection method, device and system | |
| CN112013966A (en) | Power equipment infrared image processing method based on measured temperature | |
| CN115082798A (en) | Power transmission line pin defect detection method based on dynamic receptive field | |
| CN115439573B (en) | Method for generating visible light-to-infrared image based on temperature information coding | |
| CN114155468A (en) | Method, device, equipment and medium for detecting oil leakage of transformer | |
| CN113378672A (en) | Multi-target detection method for defects of power transmission line based on improved YOLOv3 | |
| CN110070127B (en) | Household product fine identification oriented optimization method | |
| CN116994162A (en) | Unmanned aerial vehicle aerial photographing insulator target detection method based on improved Yolo algorithm | |
| CN115311186B (en) | Cross-scale attention confrontation fusion method and terminal for infrared and visible light images | |
| CN115527159B (en) | Counting system and method based on inter-modal scale attention aggregation features | |
| CN111104532A (en) | RGBD image joint recovery method based on double-current network | |
| CN117011759A (en) | Method and system for analyzing multi-element geological information of surrounding rock of tunnel face by drilling and blasting method | |
| CN116563103A (en) | Remote sensing image space-time fusion method based on self-adaptive neural network | |
| CN115331081A (en) | Image target detection method and device | |
| CN106023120B (en) | Human face portrait synthetic method based on coupling neighbour's index | |
| CN115035364A (en) | Pointer instrument reading method based on deep neural network | |
| CN111127392B (en) | No-reference image quality evaluation method based on countermeasure generation network | |
| CN115294371A (en) | Complementary feature reliable description and matching method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |