CN114519679B - Intelligent SAR target image data enhancement method - Google Patents
Intelligent SAR target image data enhancement method Download PDFInfo
- Publication number
- CN114519679B CN114519679B CN202210158438.7A CN202210158438A CN114519679B CN 114519679 B CN114519679 B CN 114519679B CN 202210158438 A CN202210158438 A CN 202210158438A CN 114519679 B CN114519679 B CN 114519679B
- Authority
- CN
- China
- Prior art keywords
- image data
- sar
- enhancement method
- filling
- target image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 238000013256 Gubra-Amylin NASH model Methods 0.000 claims abstract description 15
- 238000005070 sampling Methods 0.000 claims abstract description 5
- 230000004913 activation Effects 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 6
- 230000007547 defect Effects 0.000 claims description 3
- 238000000844 transformation Methods 0.000 claims description 3
- 230000003313 weakening effect Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 10
- 238000012549 training Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000002950 deficient Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007620 mathematical function Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012946 outsourcing Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an SAR target image data intelligent enhancement method, which comprises the following steps: inputting SAR image data to be enhanced, determining an angle of an image to be generated, and carrying out rotary filling on the SAR image data; constructing a high-resolution GAN model, which comprises a generator and a discriminator; the generator comprises in sequence: the device comprises a first convolution layer, a plurality of densely connected residual modules, a second convolution layer, an up-sampling layer and a third convolution layer; the discriminator comprises four sequence modules; and inputting the SAR image data subjected to the rotation filling into a high-resolution GAN model, and reconstructing to obtain enhanced high-resolution SAR image data. By using the intelligent data enhancement method, some SAR target data sets with small data volume, such as Mstar, can be expanded, and the classification precision is improved. The method can also be used for expanding data sets commonly used for target detection such as SSDD and the like, and improving the detection precision.
Description
Technical Field
The invention relates to the technical field of image generation, in particular to an SAR target image data intelligent enhancement method.
Background
China's aviation/aerospace reconnaissance equipment has the capabilities of multi-source, multi-band, multi-mode, multi-application and high-resolution ground imaging. However, compared with the development of imaging equipment, the on-board real-time image interpretation and emergency reconnaissance supporting information application capability are weak. The conversion from reconnaissance image data to information is one of ways to exert the operational efficiency of reconnaissance equipment and improve the utilization rate of the equipment.
The intelligent image interpretation of the missile-borne platform firstly needs to solve the problem of identification caused by the missing of a foreign military target database sample, namely the target image is deficient. For an interested foreign military target, the target is in a high-confidentiality or hidden state during non-war, sample image data of the target is extremely difficult to obtain, generally, only few image samples exist, training of a deep neural network is difficult to support, and the performance of a target recognition model is greatly limited. Therefore, for intelligent image interpretation of the missile-borne platform, firstly, intelligent enhancement technology research of the target image needs to be carried out. One method is to obtain a target characteristic library with different directions and different angles by a simulation method based on a physical model of a target from the sample enhancement angle, so as to obtain enough samples; the other method is to optimize and improve the existing model and algorithm mechanism for generating the countermeasure network from the perspective of algorithm design aiming at the problem of sample shortage so as to directly realize the direct generation of the end-to-end target image and support the data amount required by the training of the target detection model.
At present, the lack of the SAR target image data still remains one of the important reasons affecting the accuracy of the detection of a specific target. In order to fundamentally improve the robustness and reliability of SAR target detection and identification in the complex and variable battlefield environment, the problem of lack of training samples in target detection needs to be solved, and the research of SAR target image intelligent enhancement technology based on generation of countermeasure network is developed to overcome the problem of lack of training samples in SAR target detection and improve the accuracy of SAR target detection.
Aiming at important military targets of interested outsourcing, the important military targets are in a high-confidentiality state or a hidden state during non-war, sample image data of the important military targets are extremely difficult to obtain, and the important military targets generally have few image samples or even no image samples, are difficult to support training of a deep neural network, and greatly limit the target identification performance of a missile-borne platform. The existing data enhancement technology mainly has the following defects:
(1) Most of the data are subjected to geometric feature enhancement, and the enhancement technology is single.
(2) Data is enhanced mainly in the network, so that the time for training the model is increased;
(3) The deep learning black box effect is lack of interpretability, so that the learned characteristic representation is lack of physical attributes, and the practical application is difficult to carry out;
(4) The target identification model based on a single data source lacks robustness and cannot cope with mutability and diversity in a complex combat environment.
Therefore, the invention provides an SAR target image data intelligent enhancement method.
Disclosure of Invention
In order to solve the problems, the invention provides an SAR target image data intelligent enhancement method.
The invention provides the following technical scheme.
An SAR target image data intelligent enhancement method comprises the following steps:
acquiring SAR image data to be enhanced and an angle of an image to be generated;
rotating SAR image data according to an angle of an image required to be generated, and filling the SAR image data with defects after rotation;
constructing a high-resolution GAN model according to the GAN model, wherein a generator in the high-resolution GAN model sequentially comprises: the device comprises a first convolution layer, a plurality of densely connected residual modules, a second convolution layer, an up-sampling layer and a third convolution layer; the discriminator in the high-resolution GAN model sequentially comprises four sequence modules, wherein each sequence module comprises two convolution layers, two batches of regularization and two activation functions; the dimensions to which the four sequence modules are mapped are 64, 128, 256 and 512 respectively;
and inputting the filled SAR image data into a trained high-resolution GAN model, and reconstructing to obtain enhanced high-resolution SAR image data.
Preferably, the first convolutional layer convolution kernel is 3, and the step size is 1.
Preferably, the number of densely connected residual blocks is 16.
Preferably, the second convolutional layer convolution kernel is 3, and the step size is 1.
Preferably, the upsampling layer comprises a convolutional layer, an activation function and a pixel interpolation, and the pixel interpolation multiple is 2.
Preferably, the third convolutional layer convolution kernel is 3, and the step size is 1.
Preferably, the pattern of filling comprises: constant, filling the image with the characteristics of consecutive pixels; edge, filling the image with Edge weakening; symmetric, filling in images using Symmetric transformations; reflect, filling up the image using reflection transformation; wrap, fill the image with original image home position pixels.
The invention has the beneficial effects that:
the invention provides an SAR target image data intelligent enhancement method, which can expand some SAR target data sets with less data quantity, such as Mstar and the like, and improve the classification precision. The method can also be used for expanding a data set commonly used for target detection such as SSDD and the like, and improving the detection precision.
Drawings
FIG. 1 is an overall flow diagram of an embodiment of the invention;
FIG. 2 is a comparison diagram of residual modules for an embodiment of the present invention;
FIG. 3 is a diagram of a generator network architecture according to an embodiment of the present invention;
fig. 4 is a diagram of a network structure of an authenticator according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
Example 1
The invention discloses an SAR target image data intelligent enhancement method, which comprises the following steps as shown in figures 1-4:
s1: inputting SAR image data to be enhanced, determining an angle of an image to be generated, and carrying out rotary filling on the SAR image data;
s2: constructing a high-resolution GAN model, comprising a generator and a discriminator, training the high-resolution GAN model by using an SAR image data set, and storing the best weight of training; the SAR image dataset is acquired SAR image data, or one of existing SAR image datasets, such as an Mstar, SSDD, or other SAR target dataset.
The Generator (Generator Network) comprises in sequence: a first convolutional layer, a plurality of densely connected residual error modules (RRDB), a second convolutional layer, an upsampling layer, and a third convolutional layer; the first convolution layer convolution kernel is 3, and the step length is 1; 16 densely connected residual modules; the second convolutional layer convolution kernel is 3, and the step length is 1; the up-sampling layer comprises a convolution layer, an activation function (LeakyReLU) and a pixel interpolation (PixelShuffle), and the pixel interpolation multiple is 2; the third convolutional layer convolution kernel is 3, step size is 1. Wherein, the residual module pair is shown in fig. 2, and the generator network structure is shown in fig. 3.
The Discriminator (Discriminator Network) comprises four sequence modules; each sequence module comprises: convolutional layer with 3 convolution kernels and step size of 1, batch regularization (BN), activation function (LeakyReLU), convolutional layer with 3 convolution kernels and step size of 2, batch regularization (BN), activation function (LeakyReLU). The dimension to which the pictures of the first sequence module of the discriminator are mapped is 64, and the second to fourth modules are mapped to 128, 256 and 512 respectively. Wherein the network structure of the discriminator is shown in fig. 4.
After the picture is input into the generator, the generator outputs the picture and sends the picture into the discriminator, the discriminator compares the picture of the generator with the real picture, judges whether the picture given by the generator is the real picture, then the generation model improves the generator according to the discrimination model to generate a new picture, and then the new picture is judged by the discriminator. The game scene continues until the generated model and the discrimination model can not promote the game, so that the generated model becomes a perfect model.
S3: and inputting the SAR image data subjected to the rotation filling into a high-resolution GAN model, and reconstructing to obtain enhanced high-resolution SAR image data.
In the present embodiment, the first and second electrodes are,
and inputting an SAR target image, converting the image into an array matrix, and detecting the width and height of the input image. Divide the width and height by 2 and round down to get the center point of the picture. And (3) calculating the rotation radian according to the angle of the image required to be generated by input, and then processing through a mathematical function (math) to obtain the image with the required angle. However, the four corners of the rotated picture may be defective, and thus the image needs to be filled. The filling modes mainly include the following 5 modes: constant, filling the image with the characteristics of consecutive pixels; edge, filling the image with Edge weakening; symmetric, filling in images using Symmetric transformations; reflect, filling the image using reflection transformation; wrap, fill the image with original image home position pixels.
And selecting a mode of filling the image according to the actual situation according to the difference of the size ratio and the position of the target. The target occupies a small proportion of the image, a Wrap mode is generally adopted, and an Edge mode is generally adopted when the target is larger. And sending the generated image with the specified angle into a newly trained high-resolution GAN (ESRGAN) model to finally obtain an image with any angle subjected to high-resolution enhancement, thereby achieving the purpose of expanding a data set.
The present invention is not limited to the above preferred embodiments, and any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (7)
1. An SAR target image data intelligent enhancement method is characterized by comprising the following steps:
acquiring SAR image data to be enhanced and an angle of an image to be generated;
rotating SAR image data according to an angle of an image required to be generated, and filling the SAR image data with defects after rotation;
constructing a high-resolution GAN model according to the GAN model, wherein a generator in the high-resolution GAN model sequentially comprises: the device comprises a first convolution layer, a plurality of densely connected residual modules, a second convolution layer, an up-sampling layer and a third convolution layer; the discriminator in the high-resolution GAN model sequentially comprises four sequence modules, wherein each sequence module comprises two convolution layers, two batches of regularization and two activation functions; the dimensions to which the four sequence modules are mapped are 64, 128, 256 and 512 respectively;
and inputting the filled SAR image data into a trained high-resolution GAN model, and reconstructing to obtain enhanced high-resolution SAR image data.
2. The SAR target image data intelligent enhancement method of claim 1, characterized in that the first convolution layer convolution kernel is 3, and the step size is 1.
3. The SAR target image data intelligent enhancement method of claim 1, characterized in that the number of the densely connected residual modules is 16.
4. The SAR target image data intelligent enhancement method of claim 1, characterized in that the second convolutional layer convolution kernel is 3, and the step size is 1.
5. The SAR target image data intelligent enhancement method of claim 1, characterized in that the up-sampling layer comprises a convolution layer, an activation function, a pixel interpolation, and the pixel interpolation multiple is 2.
6. The SAR target image data intelligent enhancement method of claim 1, characterized in that the third convolutional layer convolutional kernel is 3, and the step size is 1.
7. The SAR target image data smart enhancement method of claim 1, characterized in that the populated pattern comprises: constant, filling the image with the characteristics of consecutive pixels; edge, filling the image with Edge weakening; symmetric, filling in images using Symmetric transformations; reflect, filling up the image using reflection transformation; wrap, fill the image with original image home position pixels.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210158438.7A CN114519679B (en) | 2022-02-21 | 2022-02-21 | Intelligent SAR target image data enhancement method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210158438.7A CN114519679B (en) | 2022-02-21 | 2022-02-21 | Intelligent SAR target image data enhancement method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114519679A CN114519679A (en) | 2022-05-20 |
| CN114519679B true CN114519679B (en) | 2022-10-21 |
Family
ID=81598402
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210158438.7A Active CN114519679B (en) | 2022-02-21 | 2022-02-21 | Intelligent SAR target image data enhancement method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114519679B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114866162B (en) * | 2022-07-11 | 2023-09-26 | 中国人民解放军国防科技大学 | Signal data enhancement method and system and communication radiation source identification method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112001847A (en) * | 2020-08-28 | 2020-11-27 | 徐州工程学院 | Method for generating high-quality image by relatively generating antagonistic super-resolution reconstruction model |
| CN112699717A (en) * | 2019-10-23 | 2021-04-23 | 中国科学院遥感与数字地球研究所 | SAR image generation method and generation device based on GAN network |
| CN113962360A (en) * | 2021-10-09 | 2022-01-21 | 西安交通大学 | Method and system for enhancing sample data based on GAN network |
| CN113962883A (en) * | 2021-09-30 | 2022-01-21 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Method for enhancing ship target detection SAR image data |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018053340A1 (en) * | 2016-09-15 | 2018-03-22 | Twitter, Inc. | Super resolution using a generative adversarial network |
| CN110555811A (en) * | 2019-07-02 | 2019-12-10 | 五邑大学 | SAR image data enhancement method and device and storage medium |
-
2022
- 2022-02-21 CN CN202210158438.7A patent/CN114519679B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112699717A (en) * | 2019-10-23 | 2021-04-23 | 中国科学院遥感与数字地球研究所 | SAR image generation method and generation device based on GAN network |
| CN112001847A (en) * | 2020-08-28 | 2020-11-27 | 徐州工程学院 | Method for generating high-quality image by relatively generating antagonistic super-resolution reconstruction model |
| CN113962883A (en) * | 2021-09-30 | 2022-01-21 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Method for enhancing ship target detection SAR image data |
| CN113962360A (en) * | 2021-10-09 | 2022-01-21 | 西安交通大学 | Method and system for enhancing sample data based on GAN network |
Non-Patent Citations (2)
| Title |
|---|
| Real-World DEM Super-Resolution Based on Generative Adversarial Networks for Improving InSAR Topographic Phase Simulation;Zherong Wu等;《IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing》;20210818;8373-8385 * |
| 面向SAR图像处理的生成式对抗网络应用综述;卢庆林等;《电讯技术》;20200131;121-128 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114519679A (en) | 2022-05-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111652321B (en) | Marine ship detection method based on improved YOLOV3 algorithm | |
| US20210183022A1 (en) | Image inpainting method and apparatus, computer device, and storage medium | |
| US11640690B2 (en) | High resolution neural rendering | |
| CN112668494A (en) | Small sample change detection method based on multi-scale feature extraction | |
| CN111862289B (en) | Point cloud up-sampling method based on GAN network | |
| CN103578093B (en) | Method for registering images, device and augmented reality system | |
| CN111028327A (en) | Three-dimensional point cloud processing method, device and equipment | |
| CN112927359A (en) | Three-dimensional point cloud completion method based on deep learning and voxels | |
| CN113538246B (en) | Remote sensing image super-resolution reconstruction method based on unsupervised multi-stage fusion network | |
| CN113159232A (en) | Three-dimensional target classification and segmentation method | |
| CN111414953A (en) | Point cloud classification method and device | |
| CN111027576A (en) | Cooperative significance detection method based on cooperative significance generation type countermeasure network | |
| CN114519679B (en) | Intelligent SAR target image data enhancement method | |
| CN111860124A (en) | Remote sensing image classification method based on space spectrum capsule generation countermeasure network | |
| CN115082322B (en) | Image processing method and device, and training method and device of image reconstruction model | |
| Jiang et al. | Local and global structure for urban ALS point cloud semantic segmentation with ground-aware attention | |
| CN117115359B (en) | Multi-view power grid three-dimensional space data reconstruction method based on depth map fusion | |
| CN112149518A (en) | Pine cone detection method based on BEGAN and YOLOV3 models | |
| CN117422619A (en) | Training method of image reconstruction model, image reconstruction method, device and equipment | |
| CN116129234A (en) | Attention-based 4D millimeter wave radar and vision fusion method | |
| CN115797489A (en) | Rock CT digital image true colorization processing method based on depth network | |
| WO2022197439A1 (en) | High resolution neural rendering | |
| Ahmadibeni | Aerial Vehicles Automated Target Recognition of Synthetic SAR Imagery Using Hybrid Stacked Denoising Autoencoders | |
| Tuia et al. | Deep Learning‐based Semantic Segmentation in Remote Sensing | |
| Lange | Translation of remote sensing images for the classification of unlabeled SAR data using Deep Convolutional Generative Adversarial Networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |