CN111931553B - Method, system, storage medium and application for enhancing generation of remote sensing data into countermeasure network - Google Patents
Method, system, storage medium and application for enhancing generation of remote sensing data into countermeasure network Download PDFInfo
- Publication number
- CN111931553B CN111931553B CN202010496962.6A CN202010496962A CN111931553B CN 111931553 B CN111931553 B CN 111931553B CN 202010496962 A CN202010496962 A CN 202010496962A CN 111931553 B CN111931553 B CN 111931553B
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- image
- generation
- generator
- countermeasure network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000003860 storage Methods 0.000 title claims abstract description 8
- 230000002708 enhancing effect Effects 0.000 title claims description 8
- 238000012545 processing Methods 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims description 22
- 230000006870 function Effects 0.000 claims description 17
- 230000011218 segmentation Effects 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 2
- 238000013434 data augmentation Methods 0.000 claims 5
- 230000008569 process Effects 0.000 abstract description 15
- 238000013528 artificial neural network Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 9
- 238000001514 detection method Methods 0.000 description 8
- 238000013135 deep learning Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 238000009826 distribution Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001965 increasing effect Effects 0.000 description 2
- GMBQZIIUCVWOCD-WWASVFFGSA-N Sarsapogenine Chemical compound O([C@@H]1[C@@H]([C@]2(CC[C@@H]3[C@@]4(C)CC[C@H](O)C[C@H]4CC[C@H]3[C@@H]2C1)C)[C@@H]1C)[C@]11CC[C@H](C)CO1 GMBQZIIUCVWOCD-WWASVFFGSA-N 0.000 description 1
- RTAQQCXQSZGOHL-UHFFFAOYSA-N Titanium Chemical compound [Ti] RTAQQCXQSZGOHL-UHFFFAOYSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000003042 antagnostic effect Effects 0.000 description 1
- 230000008485 antagonism Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention belongs to the technical field of remote sensing image processing, and discloses a method, a system, a storage medium and application of a remote sensing data enhancement generation countermeasure network. The multiple convolutions of the image in the downsampling process are important factors causing semantic loss of the image, so that the invention provides an improved downsampling module which effectively reduces the semantic loss of the image. The method improves the image generation speed and solves the problem of long time consumption of the algorithm. For the deep neural network, the longer the running time of the algorithm is, the shorter the time consumption of the network with smaller parameter quantity is, and the method provided by the invention is inspired by dividing the generation model into a plurality of sub-networks with similar structures, and under the condition that the generation quality is not great, the sub-network with smaller parameter quantity is selected as the generation model, so that the generation speed of the image is effectively improved.
Description
Technical Field
The invention belongs to the technical field of remote sensing image processing, and particularly relates to a method, a system, a storage medium and application of a remote sensing data enhancement generation countermeasure network.
Background
At present, with the development of artificial intelligence, in the field of remote sensing image processing, professionals can finish tasks such as image classification, detection and the like by means of a deep learning algorithm. The classification and detection results of the remote sensing images can be used in many aspects, such as detection of illegal buildings, detection of land use type variations. However, the remote sensing image is difficult to acquire and is affected by interference factors such as orbit period in the shooting process, which causes such a contradiction: the number of samples of the remote sensing image cannot meet the training requirements of the deep learning algorithm. Data enhancement techniques can effectively address this problem. The original data enhancement techniques included: clipping, scaling, color conversion, flipping, etc., which, while increasing the number of samples, do not change the diversity of the samples and result in lower image quality. In recent years, a depth generation model is widely used as a new data enhancement technology in the field of image processing. The depth generation model not only can increase the number of samples, but also has high generated sample quality and diversity. The deep learning algorithm has high requirements on the resolution and diversity of the remote sensing image samples, and the adoption of the depth generation model for sample number expansion is the best scheme for solving the contradiction.
The current depth generation model mainly comprises two types: GAN (generated against network, generative adversarial network) and VAE (variational self-encoder, variational autoencoder). Compared with VAE, GAN does not need to specify data distribution, the training process is clear, and the generated image quality is higher. Therefore, GAN is selected as the basic generative model.
The prior art proposes an algorithm for generating images from images based on a condition generating countermeasure network. The algorithm learns the mapping G between the observed image x and the random noise z to the output image y: { x, z } - > y, thereby completing the generation of an image. The algorithm comprises a generator and a discriminator, wherein the generator is used for generating an image, and the discriminator is used for discriminating the true and false of the generated image. The discriminators constantly learn to distinguish between real images and generated images, and the generator constantly learns to mask the discriminators. When the arbiter cannot distinguish whether the input image is a real image or a generated image, the generator can be used for image generation. The semantic information of the image generated by the prior art is lost greatly, the semantic information of a high layer is continuously lost in the repeated downsampling process of the image, the quality of the generated image is low, and great challenges are brought to the classification and detection tasks of the image. The generation speed of the image is low, and under the condition that the number of the generated samples is large, the prior technical scheme consumes a long time, and when the data amount is too large, the generation task is few, namely days and weeks, so that a large amount of time and cost are wasted.
Through the above analysis, the problems and defects existing in the prior art are as follows:
(1) The semantic information loss of the prior art image is large.
(2) The generation speed of the prior art image is slow, and a great deal of time and cost are wasted.
The difficulty of solving the problems and the defects is as follows:
1. the problem of how to design a new downsampling module is one of the difficulties in reducing the loss of high-level semantic information caused by multiple downsampling of images and improving the downsampling process of the prior art.
2. How to optimize the network structure of the generated model in the current technology, and reducing the network parameter quantity are difficult problems of improving the image generation speed.
The meaning of solving the problems and the defects is as follows:
1. the quality of the generated image can be improved, the accuracy of the image classification and detection tasks is increased, and the method is beneficial to professionals to complete the classification and detection tasks in the related fields more quickly.
2. The image generation speed can be improved, more generation samples can be obtained within a limited time, researchers do not need to spend a great deal of time waiting for the end of the generation process, and the time cost is greatly saved.
3. The method can realize rapid generation of high-quality samples under the condition of fewer training samples, and solve the trouble that professionals cannot develop related work due to scarcity of the samples.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention provides a method, a system, a storage medium and an application for enhancing generation of remote sensing data to an countermeasure network.
The invention is realized in such a way that the method for generating the countermeasure network by enhancing the remote sensing data comprises the following steps:
first, a generator and a discriminator are constructed for generating and discriminating an image, respectively.
Secondly, constructing a loss function, and guiding training and learning of the countermeasure network model;
third, during training, the generator attempts to decrease the loss function, while the arbiter attempts to increase the loss function; when the generator and the discriminator reach Nash equilibrium points, the generator generates samples;
fourth, the generator generates a sample, and the discriminator discriminates the fidelity of the generated sample.
Further, the main structure of the generator is based on a Unet++ network, and the shallow layer features and the deep layer features adopt a long-short connection mode.
Further, the countermeasure network model includes: generator G, arbiter D, downsampling module.
Further, the shallow layer features and the deep layer features of the generator G adopt a short connection mode or a long and short connection mode.
Further, the network structures of the plurality of discriminators D are the same, and participate in training at the same time.
Further, the downsampling module comprises a data normalization layer, a convolution layer, a feature map splicing layer and an activation layer.
Further, the loss function:
L cGAN (G,D)＝E x,y [logD(x,y)]+E x,z [log(1-D(x,G * (x,z)))]；
wherein D (x, y) represents the discrimination result of the discriminator on the real sample, D (x, G) * (x, z)) represents the discrimination result of the discriminator on the generated sample.
Further, D (x, G) * (x, z)) is expressed by the following formula:
D(x,G * (x,z))＝λ 1 D(x,G(x,z) 1 )+λ 2 D(x,G(x,z) 2 )+λ 3 D(x,G(x,z) 3 )+λ 4 D(x,G(x,z) 4 )；
wherein lambda is i (i takes 1,2,3, 4) represents the weight of the ith sub-network, lambda 1 +λ 2 +λ 3 +λ 4 =1 and λ 1 ＜λ 2 ＜λ 3 ＜λ 4 。
Further, the training includes: g represents a generator for generating samples, having a plurality of outputs, using G (x, z) k (k represents 1,2,3, 4) a generated sample representing the kth subnetwork output; d represents a discriminator for judging whether the input sample is a generated sample; z represents noise subject to Gaussian distribution, x represents a sample segmentation map, and y represents a real sample; and inputting a sample segmentation graph and random noise to obtain a vivid generated sample, and completing the task of data enhancement.
Another object of the present invention is to provide a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
first, a generator and a discriminator are constructed for generating and discriminating an image, respectively.
Secondly, constructing a loss function, and guiding training and learning of the countermeasure network model;
third, during training, the generator attempts to decrease the loss function, while the arbiter attempts to increase the loss function; when the generator and the discriminator reach Nash equilibrium points, the generator generates samples;
fourth, the generator generates a sample, and the discriminator discriminates the fidelity of the generated sample.
Another object of the present invention is to provide a remote sensing data enhancement generation countermeasure network system for operating the remote sensing data enhancement generation countermeasure network method, the remote sensing data enhancement generation countermeasure network system comprising:
a generator for generating a sample;
and the discriminator is used for discriminating the fidelity degree of the generated sample.
Further, the remote sensing data enhancement generation countermeasure network system further includes: a downsampling module;
the downsampling modules are respectively from top to bottom: the device comprises a data normalization layer, a convolution layer, a feature map splicing layer and an activation layer;
the generator is connected to a plurality of discriminators, as shown in fig. 3, which is also a key that can increase the image generation speed.
The invention further aims to provide a remote sensing image processing terminal, which is provided with the remote sensing data enhancement generation countermeasure network system.
By combining all the technical schemes, the invention has the advantages and positive effects that: aiming at the problem that the number of remote sensing image samples is too small to meet the training requirement of a deep learning algorithm, the invention provides a novel generation countermeasure network algorithm D-sGAN for data enhancement of the existing samples. Aiming at the problems existing in the prior art, the invention mainly aims at: firstly, semantic information loss of the generated image is reduced, so that the quality of the generated image is improved; secondly, the image generation speed is improved, and the problem of long algorithm time consumption is solved.
The invention provides an antagonistic network algorithm D-sGAN for generating remote sensing image data enhancement. The D-sGAN can effectively reduce semantic loss of the image, generate more realistic samples, and greatly improve the generation speed of the image while guaranteeing the quality of the image. How to reduce semantic loss of images. The novel downsampling module is provided, a feature map splicing layer is added after a convolution layer, and an image segmentation map is used for supervising the feature map, so that semantic loss caused by a downsampling process is reduced. How to increase the generation speed while ensuring the generation quality. Dividing the whole generation countermeasure network into sub-networks with similar structures, comparing the image generation quality of each sub-network, and selecting the sub-network with smaller network parameters for image generation under the condition that the generation quality is close.
The invention reduces the semantic information loss of the generated image and improves the quality of the generated image. By adding the image segmentation map to conduct supervision in the downsampling process, semantic loss in the downsampling process is corrected. Meanwhile, the generator network adopts a long and short connected Unet++ structure, and combines the features of the shallow layer and the deep layer, so that the semantic information loss of the image is reduced, and the quality of the generated image is improved. The method improves the image generation speed and solves the problem of long time consumption of the algorithm. For the deep neural network, the longer the running time of the algorithm is, the shorter the time consumption of the network with smaller parameter quantity is, and the method provided by the invention is inspired by dividing the generation model into a plurality of sub-networks with similar structures, and under the condition that the generation quality is not great, the sub-network with smaller parameter quantity is selected as the generation model, so that the generation speed of the image is effectively improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly explain the drawings needed in the embodiments of the present application, and it is obvious that the drawings described below are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for enhancing generation of remote sensing data into an countermeasure network according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of a remote sensing data enhancement generation countermeasure network system according to an embodiment of the present invention;
in the figure: 1. a generator; 2. and a discriminator.
FIG. 3 is a schematic diagram of a generating countermeasure network model according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of a training process according to an embodiment of the present invention. G represents a generator for generating samples, having a plurality of outputs, using G (x, z) k (k represents 1,2,3, 4) a generated sample representing the kth subnetwork output; d represents a discriminator for judging whether the input sample is a generated sample; z represents noise subject to Gaussian distribution, x represents a sample segmentation map, and y represents a real sample; and inputting a sample segmentation graph and random noise to obtain a vivid generated sample, and completing the task of data enhancement.
Fig. 5 is a schematic diagram of a downsampling module according to an embodiment of the present invention.
Fig. 6 is a schematic diagram of a different sub-network according to an embodiment of the present invention.
FIG. 7 is a schematic diagram showing the results of the production of D-sGAN (1) and D-sGAN (2).
Detailed Description
The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In view of the problems existing in the prior art, the invention provides a method, a system, a storage medium and an application for enhancing generation of remote sensing data against a network, and the invention is described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the method for enhancing and generating the countermeasure network by using the remote sensing data provided by the invention comprises the following steps:
s101: and the construction generator and the discriminator are respectively used for generating and discriminating the image.
S102: constructing a loss function, and guiding training and learning of an countermeasure network model;
s103: during training, the generator attempts to reduce the loss function, while the arbiter attempts to increase the loss function; when the generator and the discriminator reach Nash equilibrium points, the generator generates samples;
s104: the generator generates a sample, and the discriminator discriminates the fidelity of the generated sample.
As shown in fig. 2, the remote sensing data enhancement generation countermeasure network system provided by the present invention includes:
a generator 1 for generating samples.
And a discriminator 2 for discriminating the fidelity of the generated sample.
The technical scheme of the invention is further described below with reference to the accompanying drawings.
GAN: the generation of the countermeasure network (generative adversarial network) is a method for training the classifier in a semi-supervised mode, can help solve the problem of less labeled training set samples, and consists of a generator and a discriminator.
VAE: the variation is derived from an encoder (variational autoencoder), an unsupervised learning model that learns complex distributions.
Unet: the deep learning network algorithm, which was originally used for image segmentation, is now also used for image classification and detection tasks.
Unet++: improved deep learning network algorithms are structurally performed in the uiet.
D-sGAN: the deep-supervised generation countermeasure network (GAN) is the name of the generation countermeasure network algorithm proposed by the present invention.
The invention provides a new generation antagonism network algorithm D-sGAN for data enhancement. The overall schematic diagram of the generation of the countermeasure network model proposed by the invention is shown in fig. 3: in fig. 3, G represents a generator, D represents a discriminator, a gray block represents a downsampling module of the present invention, a dashed arrow represents long and short connections between feature maps, and an oblique arrow represents an upsampling process. The main structure of the generator G for generating the countermeasure network in fig. 3 is based on the network++, and the shallow features and the deep features are connected in a long-short manner, but it is also possible to implement the invention in a short-connection manner.
The invention provides a generation countermeasure network algorithm D-sGAN, which comprises a generator G and a plurality of discriminators D with identical structures. The generator is used for generating samples, and the discriminator is used for discriminating the fidelity of the generated samples. In order to enable the whole model to generate images with higher quality, the lower loss function is constructed and used for guiding the training learning process of the model.
L cGAN (G,D)＝E x,y [logD(x,y)]+E x,z [log(1-D(x,G * (x,z)))]；
Wherein D (x, y) represents the discrimination result of the discriminator on the real sample, D (x, G) * (x, z)) represents the discrimination result of the discriminator on the generated sample. Considering that the proposed generation of the countermeasure network algorithm D-sGAN comprises a plurality of discriminators, D (x, G * (x, z)) can be expressed by the following formula:
D(x,G * (x,z))＝λ 1 D(x,G(x,z) 1 )+λ 2 D(x,G(x,z) 2 )+λ 3 D(x,G(x,z) 3 )+λ 4 D(x,G(x,z) 4 )；
wherein lambda is i (i takes 1,2,3, 4) represents the weight of the ith sub-network, lambda 1 +λ 2 +λ 3 +λ 4 =1 and λ 1 ＜λ 2 ＜λ 3 ＜λ 4 。
During training, the generator attempts to reduce the loss function, while the arbiter attempts to increase the loss function. When the generator and the arbiter reach the nash equilibrium point, the generator can be used to perform the generation of samples.
The overall training flow diagram for generating the countermeasure network D-sGAN is shown in fig. 4. In FIG. 4, G represents a generator for generating samples having a plurality of outputs, G (x, z) k (k represents 1,2,3, 4) representing the generated samples of the kth subnetwork output. D represents a discriminator for judging whether the input sample is a generated sample. z represents noise subject to gaussian distribution, x represents a sample segmentation map, and y represents a real sample. According to the flow chart of fig. 4, only the sample segmentation graph and random noise are required to be input, so that a vivid generated sample can be obtained, and the task of data enhancement is completed.
The technical scheme of the invention is further described below with reference to specific embodiments.
Example 1: the semantic information loss of the generated image is reduced.
In the prior art, the semantic loss of an image is caused by multiple scale transformations of the image during downsampling. Therefore, the invention provides a novel downsampling module which can effectively reduce semantic loss. A schematic of the downsampling module is shown in fig. 5.
In fig. 5, the downsampling modules are designed as follows from top to bottom: the device comprises a data normalization layer, a convolution layer, a feature map splicing layer and an activation layer. Compared with the original downsampling process, the feature map stitching layer is newly added behind the convolution layer, and the segmentation map of the remote sensing image is used for supervising the corresponding feature map, so that semantic loss after image convolution is reduced.
Example 2: improving the image generation speed and reducing the time consumption of image generation
The generator in the generation countermeasure network D-sGAN proposed by the present invention includes a plurality of discriminators D, as shown in fig. 3 above, which is also a key that can increase the image generation speed. The pair in FIG. 3The anti-generation network comprises four sub-networks, D-sGAN L 1 ，D-sGAN L 2 ，D-sGAN L 3 ，D-sGAN L 4 A detailed schematic diagram of a subnetwork is shown in fig. 6.
In the prior art, the process of generating a model without a regional molecular network is shown in FIG. 6, namely, the model is generated only by D-sGAN L 4 The network, this results in an excessive amount of parameters for generating the model, and the speed of generating the image is greatly reduced.
The invention divides the network of the generating model into a plurality of sub-networks with similar structures. During the training process, if the sub-network D-sGAN L 1 And a subnetwork D-sGAN L 4 The quality of the output result of (a) is not much different, then the sub-network D-sGAN L 1 Can replace the sub-network D-sGAN L 4 Image generation is performed, similarly, if the subnetwork D-sGAN L 2 、D-sGAN L 3 And a subnetwork D-sGAN L 4 The quality of the output result of (a) is not much different, then the sub-network D-sGAN L 2 、D-sGAN L 3 Can replace the sub-network D-sGAN L 4 Image generation is performed. Due to the subnetwork D-sGAN L 1 ，D-sGAN L 2 ，D-sGAN L 3 The network parameter quantity of (2) is far lower than D-sGAN L 4 Therefore, the image generation speed is greatly improved while the image generation quality is ensured.
In example 1 of the present invention, an improved downsampling module is mentioned, and in order to verify the effectiveness of the downsampling module, the present invention compares the effects of two generation countermeasure networks (D-sGAN (1), D-sGAN (2)) on the test set. The two networks differ in that: D-sGAN (2) uses the improved downsampling module of the present invention, and D-sGAN (1) does not use the improved downsampling module of the present invention. The effect comparison of the two is shown in table 1 below.
TABLE 1 comparison of D-sGAN (1) and D-sGAN (2) effects
The Class IOU in the table represents the classification gain of the image, and in general, the more semantic information the image contains, the greater the value of the Class IOU. From the table, the classification gain of D-sGAN (2) is larger than that of D-sGAN (1), which indicates that the downsampling module provided by the invention is helpful for reducing the semantic information loss of images.
The generation quality of the image can reflect the effectiveness of the downsampling module provided by the invention, and the invention compares the generation results of the D-sGAN (1) and the D-sGAN (2) on the test set, as shown in fig. 7, and the generation results of the D-sGAN (1) and the D-sGAN (2) in fig. 7 are shown in fig. D.
As can be seen from fig. 7, the samples generated by D-sGAN (2) are clearer than the samples generated by D-sGAN (1), which indicates that the downsampling module according to the present invention can improve the quality of the generated image.
To demonstrate that the present invention can reduce the semantic information loss of the generated image, the present invention compares the proposed effect of generating the countermeasure network algorithm D-sGAN with that of several existing generating countermeasure network algorithms (CoGAN, simGAN, cycleGAN), as shown in table 2.
TABLE 2 results of comparison of the invention with the prior art
In table 2, class IOU represents the classification gain of an image, and in general, the more semantic information an image contains, the larger the value of Class IOU. In the table, the Class IOU of D-sGAN is significantly larger than other techniques, and this result demonstrates: compared with the prior art, the generation of the countermeasure network algorithm D-sGAN can reduce semantic information loss of generated images and achieve higher image classification accuracy.
In order to prove that the invention can improve the generation speed of images and reduce the generation time consumption, the invention compares the effects of the proposed generation of several sub-networks (D-sGAN L (1), D-sGAN L (2), D-sGAN L (3) and D-sGAN L (4)) of the countermeasure network algorithm D-sGAN, as shown in the table 3.
TABLE 3 Generation effects of different subnetworks
The model of the test equipment used by the invention is NVIDIA TITAN Xp of a 12GB memory, and the size of a test data set is 20k. The structure of the different subnetworks is shown in fig. 6. In the table, information time represents the time taken by the different sub-networks to generate the image, and Class IOU represents the classification gain of the image. Taking the subnetworks D-sGAN L (3) and D-sGAN L (4) as examples, the D-sGAN L (3) reduces the generation time consumption of 20s under the condition of losing the classification gain of 0.01. This result illustrates: under the condition that the classification gains are not different, the sub-network divided by the method can reduce the time consumption of generation and improve the image generation speed.
In the description of the present invention, unless otherwise indicated, the meaning of "a plurality" is two or more; the terms "upper," "lower," "left," "right," "inner," "outer," "front," "rear," "head," "tail," and the like are used as an orientation or positional relationship based on that shown in the drawings, merely to facilitate description of the invention and to simplify the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be construed as limiting the invention. Furthermore, the terms "first," "second," "third," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
It should be noted that the embodiments of the present invention can be realized in hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or special purpose design hardware. Those of ordinary skill in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such as provided on a carrier medium such as a magnetic disk, CD or DVD-ROM, a programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The device of the present invention and its modules may be implemented by hardware circuitry, such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., as well as software executed by various types of processors, or by a combination of the above hardware circuitry and software, such as firmware.
The foregoing is merely illustrative of specific embodiments of the present invention, and the scope of the invention is not limited thereto, but any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention will be apparent to those skilled in the art within the scope of the present invention.
Claims (5)
1. A method for enhancing generation of a countermeasure network by remote sensing data, the method comprising:
firstly, constructing a generator and a discriminator which are respectively used for generating and discriminating images;
secondly, constructing a loss function, and guiding training and learning of the countermeasure network model;
third, during training, the generator attempts to decrease the loss function, while the arbiter attempts to increase the loss function; when the generator and the discriminator reach Nash equilibrium points, the generator generates samples;
fourth, the generator generates a sample, and the discriminator discriminates the fidelity of the generated sample;
the challenge network model includes: a generator G, a plurality of discriminators D, a downsampling module; the shallow layer characteristic and the deep layer characteristic of the generator G adopt a short connection mode or a long and short connection mode;
the network structures of the plurality of discriminators D are the same, and the discriminators D participate in training at the same time;
the downsampling module comprises a data normalization layer, a convolution layer, a feature map splicing layer and an activation layer; after convolution, supervising the corresponding feature map by using the segmentation map of the remote sensing image;
the loss function:
L cGAN (G,D)＝E x,y [log D(x,y)]+E x,z [log(1-D(x,G * (x,z)))]；
wherein D (x, y) represents the discrimination result of the discriminator on the real sample, D (x, G) * (x, z)) represents the discrimination result of the discriminator on the generated sample;
D(x,G * (x, z)) is expressed by the following formula:
D(x,G * (x,z))＝λ 1 D(x,G(x,z) 1 )+λ 2 D(x,G(x,z) 2 )+λ 3 D(x,G(x,z) 3 )+λ 4 D(x,G(x,z) 4 )；
wherein lambda is i (i takes 1,2,3, 4) to represent the weight of the ith sub-network,
λ 1 +λ 2 +λ 3 +λ 4 =1 and λ 1 ＜λ 2 ＜λ 3 ＜λ 4 。
2. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the remote sensing data enhancement generation countermeasure network method of claim 1.
3. A remote sensing data augmentation generation countermeasure network system that operates the remote sensing data augmentation generation countermeasure network method of claim 1, the remote sensing data augmentation generation countermeasure network system comprising:
a generator for generating a sample;
and the discriminator is used for discriminating the fidelity degree of the generated sample.
4. The remote sensing data augmentation generation countermeasure network system of claim 3, wherein the remote sensing data augmentation generation countermeasure network system further comprises:
a downsampling module;
the downsampling modules are respectively from top to bottom: the device comprises a data normalization layer, a convolution layer, a feature map splicing layer and an activation layer;
the generator is connected with a plurality of discriminators.
5. A remote sensing image processing terminal, wherein the remote sensing image processing terminal is provided with the remote sensing data enhancement generation countermeasure network system according to claim 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010496962.6A CN111931553B (en) | 2020-06-03 | 2020-06-03 | Method, system, storage medium and application for enhancing generation of remote sensing data into countermeasure network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010496962.6A CN111931553B (en) | 2020-06-03 | 2020-06-03 | Method, system, storage medium and application for enhancing generation of remote sensing data into countermeasure network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111931553A CN111931553A (en) | 2020-11-13 |
| CN111931553B true CN111931553B (en) | 2024-02-06 |
Family
ID=73317110
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010496962.6A Active CN111931553B (en) | 2020-06-03 | 2020-06-03 | Method, system, storage medium and application for enhancing generation of remote sensing data into countermeasure network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111931553B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115205692B (en) * | 2022-09-16 | 2022-11-29 | 成都戎星科技有限公司 | Typical feature intelligent identification and extraction method based on generation of countermeasure network |
| CN117612020A (en) * | 2024-01-24 | 2024-02-27 | 西安宇速防务集团有限公司 | SGAN-based detection method for resisting neural network remote sensing image element change |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287800A (en) * | 2019-05-29 | 2019-09-27 | 河海大学 | A kind of remote sensing images scene classification method based on SGSE-GAN |
| AU2020100274A4 (en) * | 2020-02-25 | 2020-03-26 | Huang, Shuying DR | A Multi-Scale Feature Fusion Network based on GANs for Haze Removal |
| CN110992262A (en) * | 2019-11-26 | 2020-04-10 | 南阳理工学院 | Remote sensing image super-resolution reconstruction method based on generation countermeasure network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11024009B2 (en) * | 2016-09-15 | 2021-06-01 | Twitter, Inc. | Super resolution using a generative adversarial network |
-
2020
- 2020-06-03 CN CN202010496962.6A patent/CN111931553B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287800A (en) * | 2019-05-29 | 2019-09-27 | 河海大学 | A kind of remote sensing images scene classification method based on SGSE-GAN |
| CN110992262A (en) * | 2019-11-26 | 2020-04-10 | 南阳理工学院 | Remote sensing image super-resolution reconstruction method based on generation countermeasure network |
| AU2020100274A4 (en) * | 2020-02-25 | 2020-03-26 | Huang, Shuying DR | A Multi-Scale Feature Fusion Network based on GANs for Haze Removal |
Non-Patent Citations (2)
| Title |
|---|
| 基于生成对抗网络的单帧遥感图像超分辨率;苏健民;杨岚心;;计算机工程与应用(第12期);全文 * |
| 基于生成对抗网络的机载遥感图像超分辨率重建;毕晓君;潘梦迪;;智能系统学报(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111931553A (en) | 2020-11-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109685135B (en) | Few-sample image classification method based on improved metric learning | |
| CN111931553B (en) | Method, system, storage medium and application for enhancing generation of remote sensing data into countermeasure network | |
| CN113313657A (en) | Unsupervised learning method and system for low-illumination image enhancement | |
| CN110245683B (en) | Residual error relation network construction method for less-sample target identification and application | |
| CN112001399B (en) | Image scene classification method and device based on local feature saliency | |
| CN113627479B (en) | Graph data anomaly detection method based on semi-supervised learning | |
| CN112884758B (en) | Defect insulator sample generation method and system based on style migration method | |
| CN112270300A (en) | Method for converting human face sketch image into RGB image based on generating type confrontation network | |
| CN113743474A (en) | Digital picture classification method and system based on cooperative semi-supervised convolutional neural network | |
| CN115564983A (en) | Target detection method and device, electronic equipment, storage medium and application thereof | |
| Su et al. | EpNet: Power lines foreign object detection with Edge Proposal Network and data composition | |
| CN107564045B (en) | Stereo matching method based on gradient domain guided filtering | |
| Lin et al. | Matching cost filtering for dense stereo correspondence | |
| CN116563636A (en) | Synthetic aperture radar image generation method and system | |
| TWI803243B (en) | Method for expanding images, computer device and storage medium | |
| CN113205102B (en) | Vehicle mark identification method based on memristor neural network | |
| Guo et al. | Accurate and fast single shot multibox detector | |
| WO2022111231A1 (en) | Cnn training method, electronic device, and computer readable storage medium | |
| CN112215868B (en) | Method for removing gesture image background based on generation of countermeasure network | |
| CN112446345B (en) | Low-quality three-dimensional face recognition method, system, equipment and storage medium | |
| CN114782322A (en) | YOLOv5 model arc additive manufacturing molten pool defect detection method | |
| CN114077895A (en) | Variational self-coding model of antagonism strategy | |
| CN110751197A (en) | Picture classification method, picture model training method and equipment | |
| Phulpagar et al. | Segmentation of noisy binary images containing circular and elliptical objects using genetic algorithms | |
| CN117593215B (en) | Large-scale vision pre-training method and system for generating model enhancement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |