CN115797163A - Target data cross-domain inversion augmentation method based on remote sensing image - Google Patents

Target data cross-domain inversion augmentation method based on remote sensing image Download PDF

Info

Publication number
CN115797163A
CN115797163A CN202310101406.8A CN202310101406A CN115797163A CN 115797163 A CN115797163 A CN 115797163A CN 202310101406 A CN202310101406 A CN 202310101406A CN 115797163 A CN115797163 A CN 115797163A
Authority
CN
China
Prior art keywords
image
domain
data
representing
remote sensing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310101406.8A
Other languages
Chinese (zh)
Other versions
CN115797163B (en
Inventor
杨小冈
王思宇
申通
卢瑞涛
席建祥
朱正杰
陈璐
李云松
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rocket Force University of Engineering of PLA
Original Assignee
Rocket Force University of Engineering of PLA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rocket Force University of Engineering of PLA filed Critical Rocket Force University of Engineering of PLA
Priority to CN202310101406.8A priority Critical patent/CN115797163B/en
Publication of CN115797163A publication Critical patent/CN115797163A/en
Application granted granted Critical
Publication of CN115797163B publication Critical patent/CN115797163B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a target data cross-domain inversion augmentation method based on remote sensing images, which comprises the following steps: step 1, generating image data multi-domain conversion of a countermeasure network based on circulation; step 2, augmenting multi-domain data based on comparison learning; and 3, carrying out image migration and synthesis to obtain a multi-domain augmentation data set. The method takes a generation countermeasure network as a frame, introduces an image data multi-domain conversion method based on a cycle generation countermeasure network and a multi-domain data augmentation method based on contrast learning, transfers a visible light remote sensing image into an infrared image and an SAR image, and takes a synthetic data set as a matching reference map of the unmanned aerial vehicle, thereby realizing the navigation and positioning tasks of the unmanned aerial vehicle by using the multi-domain images in a multi-source sensor; the method has good performance and improves the precision of the positioning matching algorithm.

Description

Target data cross-domain inversion augmentation method based on remote sensing image
Technical Field
The invention belongs to the technical field of image data set preparation, relates to target data, and particularly relates to a remote sensing image-based target data cross-domain inversion augmentation method.
Background
In recent years, unmanned flying patrol devices are rapidly developed and gradually applied to multiple fields such as military reconnaissance, striking, surveying and mapping, fire rescue, electric power line patrol and the like, and the flying patrol device realizes intelligent visual navigation and positioning technology by using a multi-source image sensor carried by the flying patrol device, so that the flying patrol device becomes a current research hotspot.
With the progress of the technology, the resolution of the optical remote sensing image is continuously improved. And obtaining information of a remote target and surrounding environment by using the optical remote sensing image, thereby realizing tasks such as navigation, positioning, reconnaissance, striking and the like of the unmanned aerial vehicle.
With the improvement of the artificial intelligence technology, the future human society is inevitably a big data era of high-speed and intelligent development, and the intelligent scene matching technology becomes one of the important ways of navigation and positioning. The intelligent matching algorithm model based on deep learning is obtained by training a data set and analyzing and mining different differential information of data, so that a large number of multi-domain heterogeneous images are required to be used as data support in an early model training process, and the quality of the data set directly influences the capability of an artificial intelligent algorithm model. Most research focuses mainly on the algorithm model of artificial intelligence, but neglects that the intelligent algorithm needs a lot of data as a driver to obtain better algorithm performance.
Due to the lack of other domain image samples, navigation and positioning by using a multi-source imaging sensor are a difficult task, so that the development of a target data cross-domain inversion augmentation method based on remote sensing images is a task with great practical significance and higher difficulty.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a target data cross-domain inversion augmentation method based on remote sensing images, and solve the technical problem that the positioning accuracy of various imaging technologies in the unmanned aerial vehicle navigation positioning task in the prior art needs to be further improved.
In order to solve the technical problems, the invention adopts the following technical scheme:
a target data cross-domain inversion augmentation method based on remote sensing images comprises the following steps:
step 1, generating image data multi-domain conversion of the countermeasure network based on circulation:
step 101, generating an image of the countermeasure network based on the loop.
And 102, judging a generated image of the loop generation countermeasure network based on the loop generation countermeasure network.
Step 103, designing a total loss function between the generated image and the true value.
And 2, augmenting multi-domain data based on comparative learning.
Step 3, image migration and synthesis are carried out to obtain a multi-domain augmentation data set:
step 301, a set of visible light remote sensing image/infrared image unpaired data sets are given for use respectively
Figure SMS_1
And
Figure SMS_2
representing, and using a set of visible remote sensing image/SAR image unpaired data sets respectively
Figure SMS_3
And
Figure SMS_4
representing, given a set of remote-sensing image data of visible light to be converted
Figure SMS_5
For use as a verification set;
step 302, training the two groups of data given in step 301 respectively through the method of generating image data multi-domain conversion of the countermeasure network based on circulation in step 1, and realizing conversion of the visible light remote sensing image into a corresponding infrared image data set through model reasoning
Figure SMS_6
And SAR image dataset
Figure SMS_7
；
Step 303, training the two groups of data given in the step 301 respectively through the contrast learning-based multi-domain data augmentation method in the step 2, and realizing conversion of the visible light remote sensing image into a corresponding infrared image data set through model reasoning
Figure SMS_8
And SAR image dataset
Figure SMS_9
；
Step 304, respectively fusing the data sets obtained in the step 302 and the step 303 to form a fused infrared image data set
Figure SMS_10
And fusing the SAR image dataset
Figure SMS_11
Thereby forming a multi-domain augmented data set.
And 4, similarity calculation and matching test:
and (4) taking the multi-domain augmentation data set obtained in the step (3) as a reference image, and calculating the similarity of the image through a PSNR algorithm and an LPIPS algorithm.
And (4) taking the multi-domain augmentation data set obtained in the step (3) as a reference map, and performing matching test through an ORB algorithm and a LoFTR algorithm.
Compared with the prior art, the invention has the following technical effects:
the method comprises the steps that (I) a countermeasure network is generated as a framework, an image data multi-domain conversion method based on a cyclic generation countermeasure network and a multi-domain data augmentation method based on contrast learning are introduced, a visible light remote sensing image is transferred into an infrared image and an SAR image, a synthetic data set is used as a matching reference map of the unmanned aerial vehicle, and therefore the unmanned aerial vehicle can utilize the multi-domain image in a multi-source sensor to conduct navigation and positioning tasks; the method has good performance and improves the precision of the positioning matching algorithm.
The method of the invention does not need a training data set based on image pairs in the model training process based on the cycle generation countermeasure network and the contrast learning-based multi-domain data augmentation method, greatly reduces the difficulty of data preparation before training, and improves the image conversion efficiency.
The method of the invention converts the single-domain image into the multi-domain image, reduces the single sensor limitation of the unmanned aerial vehicle in the visual navigation, and utilizes the multi-domain image in the multi-source sensor to perform navigation positioning, thereby effectively improving the positioning precision of the aerial vehicle.
(IV) the method of the invention carries out a large amount of data generation and experimental comparison. Compared with the traditional matching algorithm and the existing intelligent matching algorithm, the multi-domain data set generated by the method improves the probability of image matching and the effectiveness is well verified.
And (V) the method prevents the overfitting problem frequently occurring in deep learning training by adding the data set, improves the precision and generalization capability of the model, and enriches the types of the different source data sets, thereby realizing the visual navigation and positioning of the multi-domain image.
Drawings
Fig. 1 is a schematic diagram of a cycle generation countermeasure network architecture.
FIG. 2 is a comparative learning generator framework diagram.
Fig. 3 (a) and fig. 3 (b) are schematic diagrams of the conversion effect of the visible light remote sensing image/the infrared image.
Fig. 4 (a) and fig. 4 (b) are schematic diagrams of the conversion effect of the visible light remote sensing image/SAR image.
Fig. 5 is a schematic diagram of a matching result of an original visible light remote sensing image/an original infrared image.
Fig. 6 is a diagram illustrating the converted infrared image/original infrared matching result.
Fig. 7 is a schematic diagram of an original visible light remote sensing image/original SAR matching result.
Fig. 8 is a diagram illustrating the SAR/original SAR matching result after conversion.
The present invention will be explained in further detail with reference to examples.
Detailed Description
It is to be understood that all devices and algorithms described in the present invention, unless otherwise specified, are intended to be implemented using any and all materials and algorithms known in the art.
In the present invention, "/" means "and", and for example, "visible light remote sensing image/SAR image" means a visible light remote sensing image and a SAR image.
SAR, known as Synthetic Aperture Radar, is a Synthetic Aperture Radar.
The invention discloses a target data cross-domain inversion augmentation method based on remote sensing images, and provides a data augmentation method which is expanded from a single domain to multiple domains to solve the problem of multi-source scene matching navigation positioning based on deep learning. According to the method, the overfitting problem frequently occurring in deep learning training is prevented by adding the data set, and the accuracy and generalization capability of the model are improved.
In the invention, various imaging technologies in the navigation and positioning task of the unmanned aerial vehicle are considered, and in order to meet the navigation and positioning requirements, the target data cross-domain inversion augmentation method based on the remote sensing image is designed to enrich the variety of different source data sets, thereby realizing the multi-domain image visual navigation and positioning.
The following embodiments are given as examples of the present invention, and it should be noted that the present invention is not limited to the following embodiments, and all equivalent changes based on the technical solutions of the present invention are included in the protection scope of the present invention.
Example (b):
the embodiment provides a target data cross-domain inversion augmentation method based on remote sensing images, which comprises the following steps:
step 1, generating image data multi-domain conversion of the confrontation network based on circulation:
the architecture of the loop generation countermeasure network is shown in fig. 1, and the loop generation countermeasure network includes three parts, namely feature extraction (i.e. encoding), image domain conversion and image reconstruction (i.e. decoding).
Step 101, generating images of the countermeasure network based on loop generation:
this step is intended to learn two domains of a given training sample
Figure SMS_14
And
Figure SMS_16
a mapping relationship between, wherein
Figure SMS_19
，
Figure SMS_13
The method comprises two generation mapping relations
Figure SMS_17
And
Figure SMS_20
first, using the generator
Figure SMS_22
So that the sample is made of
Figure SMS_15
Domain conversion to
Figure SMS_18
A domain then utilizes a generator
Figure SMS_21
So that the sample is made of
Figure SMS_23
Domain conversion to
Figure SMS_12
A domain.
In step 10101, the initial convolution operation is performed on the original image, the image size is not changed, but the feature map of the image is converted from 3 to 64.
Step 10102, two convolution layers are adopted to extract abstract features of the input image, and finally the dimension of the input image is converted from 256 × 256 × 64 to 64 × 64 × 256.
10103, using a plurality of residual error modules to combine the features
Figure SMS_24
Domain conversion to
Figure SMS_25
A domain.
Step 10104, finally, decoding is carried out by utilizing two layers of deconvolution to realize the image decoding
Figure SMS_26
Domain to
Figure SMS_27
And (4) converting the domain.
Step 102, based on the generated image discrimination of the loop generation countermeasure network:
the discriminator for generating image discrimination is a classifier based on four convolutional layers, the feature map of the input image is extracted from 3 dimensions to 512 dimensions by utilizing the convolutional layers, and then the confidence rate of the image is discriminated through a full connection layer and an average pooling layer.
Step 103, designing a total loss function between the generated image and the true value:
step 10301, resist loss function
Figure SMS_28
Applied to mapping functions
Figure SMS_29
And corresponding discriminator
Figure SMS_30
(ii) a Will fight the loss function
Figure SMS_31
Applied to mapping functions
Figure SMS_32
And corresponding discriminatorD A 。
In the formula:
Ato representAA domain;
Bto representBA domain;
Figure SMS_33
to representAA generator corresponding to the domain;
Figure SMS_34
to representBA generator corresponding to the domain;
D A to representAA discriminator corresponding to the domain;
Figure SMS_35
to representBAnd a discriminator corresponding to the domain.
Figure SMS_36
The expression of (A) is as follows:
Figure SMS_37
in the formula:
arepresenting an image;
ba true value is represented;
Figure SMS_38
denotes b is distributed over P data (b) (iii) a desire;
Figure SMS_39
denotes a distribution in P data (a) (iii) a desire;
Figure SMS_40
represents a mathematical expectation;
Figure SMS_41
is represented as distributed;
P data () Representing the probability density of the data.
In the present step, the first step is carried out,
Figure SMS_42
for generating like
Figure SMS_43
Image of a domain
Figure SMS_44
，
Figure SMS_45
For distinguishing transformed image samples
Figure SMS_46
With real samples
Figure SMS_47
。
Mapping function of this step
Figure SMS_48
And corresponding arbiterD A Introduce similar antagonism losses
Figure SMS_49
。
Step 10302, for each sheet, from
Figure SMS_50
Image of a domain
Figure SMS_51
Using a cyclic consistency loss function
Figure SMS_52
For images
Figure SMS_53
Processed, image
Figure SMS_54
One cycle should suffice to image
Figure SMS_55
Restore to original image, e.g.
Figure SMS_56
。
Said cyclic consistency loss function
Figure SMS_57
The expression is as follows:
Figure SMS_58
in the formula:
| represents the norm.
Step 10303, generate a total loss function between the image and the true value
Figure SMS_59
Expressed as:
Figure SMS_60
in the formula:
Figure SMS_61
parameters representing a cyclic loss function.
In the present embodiment, the first and second electrodes are,
Figure SMS_62
the relative importance of these two goals is controlled.
Step 2, contrast learning-based multi-domain data augmentation:
generating images based on a codec, wherein the input field of the generator for generating the images is
Figure SMS_63
The output domain is
Figure SMS_64
Giving unpaired datasets
Figure SMS_65
；
In the formula:
Figure SMS_66
representing an input field;
Figure SMS_67
representing an output domain;
Figure SMS_68
representing a set of real numbers;
Hrepresents the height of the image;
Wrepresents the width of the image;
Crepresenting the number of channels of the image;
Arepresenting unpaired datasets corresponding to input fields;
Brepresenting an unpaired data set corresponding to an output domain;
arepresenting a data setAThe data of (1);
brepresenting a data setBThe data of (1).
The image generation generator is divided into two partial coders
Figure SMS_69
And decoder
Figure SMS_70
Thereby producing an output image
Figure SMS_71
(ii) a In the present embodiment, the framework of the generator for generating the image is as shown in fig. 2. Using encoders
Figure SMS_72
And acquiring high-dimensional characteristic vectors, and performing iterative training through a total contrast loss function to realize multi-domain data augmentation.
The total contrast loss function is:
Figure SMS_73
in the formula:
Figure SMS_74
representing a resistance loss function;
Figure SMS_75
representing a maximum mutual information loss function;
Figure SMS_76
representing an external loss function;
Figure SMS_77
representing a maximum mutual information loss parameter;
Figure SMS_78
representing an external loss parameter;
Ga representation generator;
Da presentation discriminator;
Arepresenting unpaired datasets corresponding to input fields;
Brepresenting unpaired datasets corresponding to output domains;
Mrepresenting a multi-tier perceptron network.
In this embodiment, when
Figure SMS_79
，
Figure SMS_80
The method can be regarded as a lightweight version of a CycleGan network during the combined training.
In this embodiment, the penalty function is resisted
Figure SMS_81
Maximizing mutual information loss function
Figure SMS_82
And external loss function
Figure SMS_83
Are calculated by common calculation methods known in the art.
And 3, carrying out image migration and synthesis to obtain a multi-domain augmentation data set:
in step 301, a set of variables is givenRespectively using visible light remote sensing image/infrared image non-paired data sets
Figure SMS_84
And
Figure SMS_85
the representation and a group of visible light remote sensing image/SAR image unpaired data sets are respectively used
Figure SMS_86
And
Figure SMS_87
representing, given a set of remote-sensing image data of visible light to be converted
Figure SMS_88
For use as a verification set;
step 302, training the two groups of data given in step 301 respectively through the method of generating image data multi-domain conversion of the countermeasure network based on circulation in step 1, and realizing conversion of the visible light remote sensing image into a corresponding infrared image data set through model reasoning
Figure SMS_89
And SAR image dataset
Figure SMS_90
；
Step 303, training the two groups of data given in the step 301 respectively through the contrast learning-based multi-domain data augmentation method in the step 2, and realizing conversion of the visible light remote sensing image into a corresponding infrared image data set through model reasoning
Figure SMS_91
And SAR image dataset
Figure SMS_92
；
Step 304, respectively fusing the data sets obtained in the step 302 and the step 303 to form a fused infrared image data set
Figure SMS_93
And fusing SAR image datasets
Figure SMS_94
Thereby forming a multi-domain augmented data set.
In the present embodiment, the visible light remote sensing image conversion effect is as shown in fig. 3 (a), fig. 3 (b), fig. 4 (a), and fig. 4 (b).
Step 4, similarity calculation and matching test:
and (3) taking the multi-domain augmentation data set obtained in the step (3) as a reference image, and calculating the similarity of the images through a PSNR (peak-to-noise ratio) algorithm and an LPIPS (learning and perception image block similarity) algorithm.
In the step, the generation effect of the visible light remote sensing image/infrared image and the visible light remote sensing image/SAR image is evaluated through the similarity, wherein the larger the PSNR is, the smaller the LPIPS is, the higher the representative image similarity is.
In the present example, the evaluation results are shown in tables 1 and 2.
TABLE 1 comparison of conversion effect between visible light remote sensing image and infrared image
Figure SMS_95
TABLE 2 comparison of conversion effects of visible light remote sensing image/SAR image
Figure SMS_96
And (4) taking the multi-domain augmentation data set obtained in the step (3) as a reference map, and performing matching test through an ORB (brief feature point description) algorithm and a LoFTR (local feature matching) algorithm.
In this example, the test results are shown in fig. 5, 6, 7, and 8.
Simulation example:
the effects of the present invention are further illustrated by the following simulations:
1. simulation conditions are as follows:
in order to verify the effectiveness of the invention, multi-domain augmentation is carried out on a plurality of groups of data sets, and corresponding infrared and SAR image results are obtained. The experimental environment is as follows: the operating system is Ubuntu18.04, and the processor is a notebook computer with 2.9GHz IntelXeon E5-2667.
2. Simulation experiment:
the invention is used for generating a large amount of data and comparing experiments. Compared with the conventional matching algorithm and the existing intelligent matching algorithm, the multi-domain data set generated by the method improves the image matching precision and has a good effect on navigation and positioning of the unmanned aerial vehicle.
Fig. 5 shows the matching result of the original visible light remote sensing image/the original infrared image. Fig. 6 is a converted infrared image/original infrared matching result. Fig. 7 shows the original visible light remote sensing image/original SAR matching result. Fig. 8 shows the SAR/raw SAR matching result after conversion. From the figure, the multi-domain data augmentation method solves the problems of mismatching, inaccurate navigation positioning and the like, achieves navigation positioning of multi-domain images in the multi-source sensor, and effectively improves the positioning precision of the aircraft.
Comparative example 1:
the comparison example shows a target data cross-domain inversion augmentation method, and other steps of the method are basically the same as those of the embodiment except for the difference of the first step. In this comparative example, specifically:
step one, adjusting a loss function:
the loss function of the algorithm is a binary cross entropy loss function, namely, the loss function combining the binary cross entropy and the Sigmoid activation function is used for training.
Comparative example 2:
the comparison example shows a target data cross-domain inversion augmentation method, and other steps of the method are basically the same as those of the embodiment except for the difference of the first step. In this comparative example, specifically:
step one, adjusting a loss function:
the loss function of the algorithm is the Smooth L1 loss function, i.e. trained with a loss function that uses a square function around point 0 to make it smoother.
Compared with the embodiment, the comparative example 1 and the comparative example 2, the network convergence speed of the method is higher, the network stability is higher, the conversion effect after model training is shown in fig. 3 (a), fig. 3 (b), fig. 4 (a) and fig. 4 (b), and the matching experiment result shows that the loss function used by the method has a better conversion effect.
Comparative example 3:
the comparison example provides a target data cross-domain inversion augmentation method, and the method adopts a styleGAN model to perform cross-domain inversion on a remote sensing visible light remote sensing image.
Comparative example 4:
the comparison example provides a target data cross-domain inversion augmentation method, and the method adopts a Pix2Pix model to perform cross-domain inversion on a remote sensing visible light remote sensing image.
Comparing and analyzing the embodiment, the comparative example 3 and the comparative example 4, it can be found that the infrared and SAR images generated by the method have better modal consistency, and are closer to real infrared and SAR images on the basis of ensuring that the details of the image content are unchanged, while the comparative example 3 and the comparative example 4 have partial distortion.

Claims (6)

1. A target data cross-domain inversion augmentation method based on remote sensing images is characterized by comprising the following steps:
step 1, generating image data multi-domain conversion of the confrontation network based on circulation:
step 101, generating an image of a countermeasure network based on loop generation;
102, judging a generated image of the countermeasure network based on cycle generation;
103, designing a total loss function between the generated image and a true value;
step 2, augmenting multi-domain data based on comparison learning;
and 3, carrying out image migration and synthesis to obtain a multi-domain augmentation data set:
step 301, a set of visible light remote sensing image/infrared image unpairedThe data sets being used separately
Figure QLYQS_1
And
Figure QLYQS_2
representing, and using a set of visible remote sensing image/SAR image unpaired data sets respectively
Figure QLYQS_3
And
Figure QLYQS_4
representing, given a set of remote-sensing image data of visible light to be converted
Figure QLYQS_5
For use as a verification set;
step 302, training the two groups of data given in step 301 respectively through the method of generating image data multi-domain conversion of the countermeasure network based on circulation in step 1, and realizing conversion of the visible light remote sensing image into a corresponding infrared image data set through model reasoning
Figure QLYQS_6
And SAR image dataset
Figure QLYQS_7
；
Step 303, training the two groups of data given in the step 301 respectively through the contrast learning-based multi-domain data augmentation method in the step 2, and realizing conversion of the visible light remote sensing image into a corresponding infrared image data set through model reasoning
Figure QLYQS_8
And SAR image dataset
Figure QLYQS_9
；
Step 304, respectively fusing the data sets obtained in the step 302 and the step 303 to form a fused infrared imageImage data set
Figure QLYQS_10
And fusing SAR image datasets
Figure QLYQS_11
Thereby forming a multi-domain augmented data set.
2. The remote sensing image-based target data cross-domain inversion augmentation method of claim 1, wherein the step 101 comprises the following steps:
step 10101, carrying out initialization convolution operation on the original image, wherein the size of the image is unchanged, but the feature map of the image is converted from 3 to 64;
10102, extracting abstract features of the input image by adopting two convolution layers, and finally converting the dimensionality of the input image from 256 × 256 × 64 into 64 × 64 × 256;
10103, using a plurality of residual error modules to combine the features
Figure QLYQS_12
Domain conversion to
Figure QLYQS_13
A domain;
step 10104, finally, decoding is carried out by utilizing two layers of deconvolution to realize image decoding
Figure QLYQS_14
Domain to
Figure QLYQS_15
And (4) converting the domain.
3. The remote sensing image-based target data cross-domain inversion augmentation method of claim 2, wherein the step 102 comprises the following steps: the discriminator for generating image discrimination is a classifier based on four convolutional layers, extracts the feature map of the input image from 3 dimensions to 512 dimensions by using the convolutional layers, and then discriminates the confidence rate of the image through a full connection layer and an average pooling layer.
4. The remote sensing image-based target data cross-domain inversion augmentation method of claim 3, wherein step 103 comprises the following steps:
step 10301, a penalty function is run
Figure QLYQS_16
Applied to mapping functions
Figure QLYQS_17
And corresponding discriminator
Figure QLYQS_18
(ii) a Will fight against loss function
Figure QLYQS_19
Applied to mapping functions
Figure QLYQS_20
And corresponding arbiterD A ；
In the formula:
Ato representAA domain;
Bto representBA domain;
Figure QLYQS_21
to representAA generator corresponding to the domain;
Figure QLYQS_22
to representBA generator corresponding to the domain;
D A to representAA discriminator corresponding to the domain;
Figure QLYQS_23
to representBA discriminator corresponding to the domain;
step 10302, for each sheet, from
Figure QLYQS_24
Image of a domain
Figure QLYQS_25
Using a cyclic consistency loss function
Figure QLYQS_26
For images
Figure QLYQS_27
Processed, image
Figure QLYQS_28
One cycle should suffice to image
Figure QLYQS_29
Restoring to an original image;
step 10303, generate a total loss function between the image and the true value
Figure QLYQS_30
Expressed as:
Figure QLYQS_31
in the formula:
Figure QLYQS_32
parameters representing a cyclic loss function.
5. The remote sensing image-based target data cross-domain inversion augmentation method of claim 1, wherein the step 2 comprises the following steps:
image generation based on codecThe input field of the device is
Figure QLYQS_33
The output domain is
Figure QLYQS_34
Giving unpaired datasets
Figure QLYQS_35
；
In the formula:
Figure QLYQS_36
representing an input field;
Figure QLYQS_37
representing an output domain;
Figure QLYQS_38
representing a set of real numbers;
Hrepresents the height of the image;
Wrepresents the width of the image;
Crepresenting the number of channels of the image;
Arepresenting unpaired datasets corresponding to input fields;
Brepresenting unpaired datasets corresponding to output domains;
arepresenting a data setAThe data of (1);
brepresenting a data setBThe data of (1);
the image generation generator is divided into two partial coders
Figure QLYQS_39
And decoder
Figure QLYQS_40
Thereby producing an output image
Figure QLYQS_41
(ii) a Using encoders
Figure QLYQS_42
Obtaining high-dimensional characteristic vectors, and performing iterative training through a total contrast loss function to realize multi-domain data augmentation;
the total contrast loss function is:
Figure QLYQS_43
in the formula:
Figure QLYQS_44
representing a resistance loss function;
Figure QLYQS_45
representing a maximum mutual information loss function;
Figure QLYQS_46
representing an external loss function;
Figure QLYQS_47
representing a maximum mutual information loss parameter;
Figure QLYQS_48
representing an external loss parameter;
Ga representation generator;
Da presentation discriminator;
Arepresenting an unpaired data set corresponding to an input domain;
Bindication inputOut-of-domain corresponding unpaired datasets;
Mrepresenting a multi-tier perceptron network.
6. The remote sensing image-based target data cross-domain inversion augmentation method of claim 1, further comprising a step 4 of similarity calculation and matching test:
taking the multi-domain augmentation data set obtained in the step 3 as a reference image, and calculating the similarity of the image through a PSNR algorithm and an LPIPS algorithm;
and (4) taking the multi-domain augmentation data set obtained in the step (3) as a reference map, and performing matching test through an ORB algorithm and a LoFTR algorithm.
CN202310101406.8A 2023-02-13 2023-02-13 Target data cross-domain inversion augmentation method based on remote sensing image Active CN115797163B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310101406.8A CN115797163B (en) 2023-02-13 2023-02-13 Target data cross-domain inversion augmentation method based on remote sensing image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310101406.8A CN115797163B (en) 2023-02-13 2023-02-13 Target data cross-domain inversion augmentation method based on remote sensing image

Publications (2)

Publication Number Publication Date
CN115797163A true CN115797163A (en) 2023-03-14
CN115797163B CN115797163B (en) 2023-04-28

Family

ID=85430897

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310101406.8A Active CN115797163B (en) 2023-02-13 2023-02-13 Target data cross-domain inversion augmentation method based on remote sensing image

Country Status (1)

Country Link
CN (1) CN115797163B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113283444A (en) * 2021-03-30 2021-08-20 电子科技大学 Heterogeneous image migration method based on generation countermeasure network
CN113298056A (en) * 2021-07-27 2021-08-24 自然资源部国土卫星遥感应用中心 Multi-mode remote sensing image change detection method, model generation method and terminal equipment
WO2022105308A1 (en) * 2020-11-20 2022-05-27 南京大学 Method for augmenting image on the basis of generative adversarial cascaded network
CN115310515A (en) * 2022-07-06 2022-11-08 山东科技大学 Fault-labeled seismic data sample set amplification method based on generation countermeasure network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022105308A1 (en) * 2020-11-20 2022-05-27 南京大学 Method for augmenting image on the basis of generative adversarial cascaded network
CN113283444A (en) * 2021-03-30 2021-08-20 电子科技大学 Heterogeneous image migration method based on generation countermeasure network
CN113298056A (en) * 2021-07-27 2021-08-24 自然资源部国土卫星遥感应用中心 Multi-mode remote sensing image change detection method, model generation method and terminal equipment
CN115310515A (en) * 2022-07-06 2022-11-08 山东科技大学 Fault-labeled seismic data sample set amplification method based on generation countermeasure network

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
TIYASHA TIYASHA等: "Functionalization of remote sensing and on-site data for simulating surface water dissolved oxygen: Development of hybrid tree-based artificial intelligence models" *
王舒洋;慕晓冬;贺浩;杨东方;马晨晖;: "航拍图像跨数据域特征迁移道路提取方法" *
郭炜炜;张增辉;郁文贤;孙效华;: "SAR图像目标识别的可解释性问题探讨" *

Also Published As

Publication number Publication date
CN115797163B (en) 2023-04-28

Similar Documents

Publication Publication Date Title
Wu et al. Towards accurate high resolution satellite image semantic segmentation
Wang et al. Learning discriminative features by covering local geometric space for point cloud analysis
CN110263912B (en) Image question-answering method based on multi-target association depth reasoning
Li et al. Traffic scene segmentation based on RGB-D image and deep learning
WO2021057186A1 (en) Neural network training method, data processing method, and related apparatuses
CN112434628B (en) Small sample image classification method based on active learning and collaborative representation
CN111986108A (en) Complex sea-air scene image defogging method based on generation countermeasure network
CN113838064B (en) Cloud removal method based on branch GAN using multi-temporal remote sensing data
CN112036260A (en) Expression recognition method and system for multi-scale sub-block aggregation in natural environment
Zhao et al. Semantic segmentation of remote sensing image based on regional self-attention mechanism
CN114612660A (en) Three-dimensional modeling method based on multi-feature fusion point cloud segmentation
CN116612468A (en) Three-dimensional target detection method based on multi-mode fusion and depth attention mechanism
Zhou et al. Graph attention guidance network with knowledge distillation for semantic segmentation of remote sensing images
Wang et al. Generative partial multi-view clustering
CN111489405B (en) Face sketch synthesis system for generating confrontation network based on condition enhancement
CN112766217A (en) Cross-modal pedestrian re-identification method based on disentanglement and feature level difference learning
CN115797163B (en) Target data cross-domain inversion augmentation method based on remote sensing image
CN114463235A (en) Infrared and visible light image fusion method and device and storage medium
Zhao et al. Label Freedom: Stable Diffusion for Remote Sensing Image Semantic Segmentation Data Generation
Wang et al. MLAN: Multi-Level Attention Network
CN111126310A (en) Pedestrian gender identification method based on scene migration
Wei et al. An Efficient Point Cloud-based 3D Single Stage Object Detector
Li et al. Blind image quality evaluation method based on cyclic generative adversarial network
Zhao et al. Adaptive Dual-Stream Sparse Transformer Network for Salient Object Detection in Optical Remote Sensing Images
An et al. Network adaptation for color image semantic segmentation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant