CN116912680A - SAR ship identification cross-modal domain migration learning and identification method and system - Google Patents

SAR ship identification cross-modal domain migration learning and identification method and system Download PDF

Info

Publication number
CN116912680A
CN116912680A CN202310750641.8A CN202310750641A CN116912680A CN 116912680 A CN116912680 A CN 116912680A CN 202310750641 A CN202310750641 A CN 202310750641A CN 116912680 A CN116912680 A CN 116912680A
Authority
CN
China
Prior art keywords
sar
image
ship
module
optical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310750641.8A
Other languages
Chinese (zh)
Inventor
高贵
代钰曦
刘佳
姚力波
段定峰
刘涛
张晰
李恒超
郁文贤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southwest Jiaotong University
Aerospace Dongfanghong Satellite Co Ltd
Original Assignee
Southwest Jiaotong University
Aerospace Dongfanghong Satellite Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southwest Jiaotong University, Aerospace Dongfanghong Satellite Co Ltd filed Critical Southwest Jiaotong University
Priority to CN202310750641.8A priority Critical patent/CN116912680A/en
Publication of CN116912680A publication Critical patent/CN116912680A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/096—Transfer learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method and a system for identifying, transferring, learning and identifying a cross-modal domain of SAR ship identification, belongs to the technical field of artificial intelligence and synthetic aperture radar target identification, and solves the problem of poor image quality generated by image conversion in the prior art. The method comprises the steps of preprocessing an optical ship image in a source domain optical data set and an SAR ship image in a target domain SAR data set; the method comprises the steps that a dense connection feature converter and a light attention mechanism module which are connected with a feature encoder and a feature decoder are built on the basis of the feature encoder and the feature decoder of the CycleGAN to obtain a light generator network model, and an ADCG network is obtained on the basis of the light generator network model for performing an OPT2SAR task, the light generator network model for performing an SAR2OPT task and a discriminator; and training the ADCG network by the source domain optical data set and the target domain SAR data set which are obtained by preprocessing, and using the ADCG network for the pseudo SAR image of the optical ship image. The method is used for image conversion and SAR ship target recognition.

Description

SAR ship identification cross-modal domain migration learning and identification method and system
Technical Field
A method and a system for trans-modal domain migration learning and recognition of SAR ship recognition are used for image conversion and SAR ship target recognition, and belong to the technical field of artificial intelligence and synthetic aperture radar target recognition.
Background
Synthetic Aperture Radar (SAR) and optical sensors are currently the most widely used earth-looking sensors, with different imaging capabilities. The optical image has the characteristics of large sample data volume due to simple data acquisition way and easy understanding and marking of image content. The SAR is used as an active microwave remote sensing technology, is not influenced by climate and natural environment factors, has the advantages of full-time and all-weather penetrating cloud layer detection capability and long-time stable and continuous acquisition of surface information, so that the SAR has wide application in various fields, and the SAR ship automatic target recognition (Automatic Target Recognition, ATR) technology is used as an important branch of SAR image interpretation and is also widely paid attention to due to important values in the aspects of ocean monitoring, ocean transportation management and the like. With the development of deep learning, a Convolutional Neural Network (CNN) -based method, such as VGG16, resNet series, denseNet, mobileNet, inceptionNet, efficientNet and other networks, has been remarkably successful in the field of ship ATR in a data-driven manner based on the strong characteristic expression capability of CNN.
Although convolution neural network-based SAR ship target recognition methods have achieved significant success, they anyway require a large amount of marker data to train the classifier. The visual information provided by the SAR sensor is not as rich as that provided by the optical sensor, a large amount of speckle noise exists in the image, human vision cannot adapt to the microwave scattering phenomenon, and complex scattering characteristics are difficult to understand, so that expert knowledge is required to be relied on for obtaining a marked sample by interpreting the SAR image, and the target marking of a large-scale SAR ship data set is very time-consuming, so that the SAR image has the characteristics of small sample data size and unbalanced category. Under the current situation that training samples are scarce, the SAR ship ATR technology has the defects that training tasks based on a deep learning network model become very difficult, and the accuracy of ship identification is greatly affected.
In order to improve the ship recognition accuracy on the premise of not increasing the complexity of the model, the heat of the ship recognition task based on transfer learning is gradually increased. The method realizes the joint analysis of the optical ship data and the SAR ship data by utilizing the migration learning method, carries out domain knowledge transfer, can effectively solve the problems of scarcity and unbalanced category of effective marking samples of the SAR ship ATR deep learning network model, and improves the ship ATR precision.
Unfortunately, for different cross-modal domains, such as optical and SAR, significant data distribution differences between the source and target domains may not be transmitted in the lower layers, resulting in very serious domain mismatch problems. To solve the domain mismatch problem, some scholars propose a domain adaptation method to align the data of the source domain and the target domain to ensure that there is a similar probability distribution. Other scholars propose a method for implementing cross-modal domain transfer by creating a mapping, by which images in a source domain obtain characteristics of a target domain, and nonlinear differences between heterogeneous images are reduced, which is called Image-to-Image Translation, I2IT, wherein a method based on generating a countermeasure network (GAN) achieves a good effect. In this case, we aim to solve the challenges of small number of valid tag samples and class imbalance of SAR domains by transferring optical domain knowledge. Because of the significant data distribution difference between different cross-modal domains such as an optical domain and an SAR domain, the current optical ship image recognition method cannot be directly applied to SAR images, and the main problems to be solved are as follows: (1) Unlike optical images, SAR images have no color and brightness information, but mainly include texture, structure, and shape information. (2) Although the feature alignment method can reduce the discreteness of the features, the domain difference is large because of the difference between the optical and SAR imaging mechanisms, and the method is not suitable for transmission in the low-layer shared feature space.
Because of the various challenges described above, it is not easy to implement a cross-modal domain knowledge migration task from optical domain to SAR domain, but the advent of GAN provides a new idea for optical to SAR (OPT 2 SAR) image conversion. However, most of the present GAN-based image conversion methods are limited by supervised learning and conditions requiring pairs of training samples, e.g., condition generation challenge network (cGAN), pix2Pix. In practice, in the field of ship identification, paired optical image data and remote sensing image data are difficult to obtain. The unsupervised image conversion models, such as CycleGAN, UGATIT, dualGAN and DiscoGAN, do not require paired training images, and reduce the difficulty in acquiring ship slice data. The CycleGAN is used as the best known method in the unsupervised unpaired image conversion task, and the cycle consistency loss and the identity loss are introduced into the loss function of the original GAN, so that the image generated in one cycle has higher similarity with the original image, and excellent translation effect is shown in the unsupervised unpaired image conversion task.
The above-mentioned countermeasure learning method based on the CycleGAN network is used for the conversion between the optical images, realize the ship cross-modal domain knowledge migration task from the optical domain to the SAR domain, namely in the ship image conversion task of the OPT2SAR, the CycleGAN network still has some technical problems: (1) The background of the optical data center ship shore base is complex, and the rapid extraction of the main body characteristics of the ship by the CycleGAN network is interfered; and the proportion of the optical data set to the ship main body of the SAR data set in the picture is different, and the problem that the ship main body of the source domain and the target domain are different in scale can not be solved by the CycleGAN network in the image conversion process. (2) The network structure of the CycleGAN generator formed by stacking 9 residual blocks is complex, the parameter quantity is large, a large amount of training time is required to be consumed, and feature redundancy is caused. In the currently proposed two-stage ship ATR transfer learning method, in the first stage, the CycleGAN is often directly used for pixel-level domain knowledge transfer, the above problems are not considered, and the CycleGAN is used for network improvement applicable to ship ATR.
In summary, the following technical problems exist in using a CycleGAN network (i.e., directly using an open source generation countermeasure network) to realize image conversion from an optical domain to an SAR domain (i.e., to realize a task of cross-modal domain knowledge migration of the ship OPT2 SAR):
1. the image quality generated by image conversion is poor, or complex shore-based background can interfere with the image conversion of a ship main body, so that the problem of the subsequent ship classification precision is affected;
2. the network structure is complex, the parameter quantity is large, a large amount of training time is required to be consumed, and the feature redundancy is caused, so that the problem of high requirement on resource is caused.
Disclosure of Invention
The invention aims to provide a method and a system for identifying, transferring, learning and identifying across modal domains for a SAR ship, which solve the problems that in the prior art, the image quality generated by image conversion is poor, or complex shore-based background can interfere with the image conversion of a ship main body, so that the classification precision of the subsequent ship is affected.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
a SAR ship identification cross-modal domain migration learning method comprises the following steps:
s1, preprocessing an optical ship image in a source domain optical data set and an SAR ship image in a target domain SAR data set;
s2, a dense connection feature converter and a light-weight attention mechanism module which are connected with the feature encoder and the feature decoder are constructed based on the feature encoder and the feature decoder of the CycleGAN to obtain a light-weight generator network model, and an ADCG network is obtained based on the light-weight generator network model for performing an OPT2SAR task, the light-weight generator network model for performing an SAR2OPT task and the discriminator;
s3, inputting the optical ship image in the source domain optical data set and the SAR ship image in the target domain SAR data set obtained through preprocessing into an ADCG network at the same time to train the optical ship image and generating a pseudo SAR image of the optical ship image to be converted by utilizing the trained ADCG network.
Further, the specific steps of the step S1 are as follows:
s1.1, unifying all the acquired optical ship images in the source domain optical data set to 256 x 256 sizes by using a linear interpolation method;
s1.2, based on the size of each optical ship image in the source domain optical data set, firstly cutting the center of each SAR ship image in the acquired target domain SAR data set to 128 x 128 size, and then amplifying the cut image to 256 x 256 size by using a linear interpolation method.
Further, the dense connection feature converter comprises three DenseBlock blocks sequentially connected, wherein the first DenseBlock block comprises a Basicblock and two BottleneckBlock blocks sequentially connected, the Basicblock comprises a Conv layer, the BottleneckBlock comprises a 1x1Conv layer and a 3x3Conv layer sequentially connected, and Concat operation is performed on feature channels before the convolution of the 1x1Conv layer and after the convolution of the 3x3Conv layer, the DenseBlock represents a dense block, the Basicblock represents a basic block, and the BottleneckBlock represents a bottleneck block;
the second DenseBlock block and the third DenseBlock block respectively comprise three Bottleneck blocks connected in sequence.
Further, the light attention mechanism module comprises a CAM sub-module and a SAM sub-module, wherein the CAM sub-module is input by densely connecting with the output of the feature converter, the result of multiplying the SAM sub-module by the SAM sub-module with the input of the CAM sub-module is input by the feature decoder, the SAM sub-module is input by multiplying the CAM sub-module with the input of the CAM sub-module, the CAM sub-module represents the channel attention module, and the SAM sub-module represents the space attention module;
the CAM submodule comprises an average pooling layer and a maximum pooling layer, a multi-layer perceptron with a plurality of hidden layers is respectively connected with the average pooling layer and the maximum pooling layer, and element-by-element summation is used for merging output characteristic vectors of results output by the multi-layer perceptron, so that a channel attention diagram is obtained;
the SAM submodule comprises a maximum pooling layer and an average pooling layer which are sequentially connected, a convolution layer which is used for splicing the output of the average pooling layer, and a result output by the convolution layer, namely the spatial attention diagram is obtained.
Further, the expression of the game form of the lightweight generator network model and the discriminator in the ADCG network is as follows:
wherein G represents a lightweight generator network model, D represents a discriminator, i.e., a discriminator, E represents expectations, v-P r (v) Representing real data v and data distribution P thereof r (v)，z～P g (z) represents dummy data z and its specific probability distribution P g (z)，min G max D V (D, G) represents the maximum minimum game between the lightweight generator network model and the discriminator;
the total loss function of the ADCG network is expressed as:
L OPT 2SAR-GAN ＝αL GAN +βL cycle +γL identity
wherein α, β and γ each represent L GAN 、L cycle And L identity Weights of (2);
the OPT2SAR process for the formation of resistant gaming is described as follows:
wherein G is O→S Representing a lightweight generator network model in the process of forming an OPT2SAR mission-resistant game, the OPT2SAR mission representing the conversion of an optical ship image into a pseudo SAR image, D SAR Discriminator for representing a real SAR image and a pseudo SAR image, { x }, x SAR ～X SAR }、{x opt ～X opt Respectively representing a source domain optical data set and a target domain SAR data set obtained after preprocessing, x SAR And x opt Representing an optical ship image and a SAR ship image from a source domain optical dataset and a target domain SAR dataset, respectively;
the periodic consistency loss function is defined as:
wherein G is S→O (G O→S (x opt ))≈x opt And G O→S (G S→O (x SAR ))≈x SAR ，G S→O A lightweight generator network model in the SAR20PT task resistance game forming process is represented, and the SAR20PT task represents the conversion of an SAR ship image into a pseudo optical image;
the identity loss function is:
a SAR ship identification cross-modal domain migration identification method expands a target domain SAR data set of an obtained pseudo SAR image, trains VGG16, resNet50 and Mob i l eNet based on the expanded target domain SAR data set, and carries out SAR ship target identification in a to-be-identified SAR ship image.
A SAR ship identification cross-modal domain transfer learning system, comprising:
and a pretreatment module: preprocessing an optical ship image in a source domain optical data set and an SAR ship image in a target domain SAR data set;
ADCG network construction module: the method comprises the steps that a dense connection feature converter and a light attention mechanism module which are connected with a feature encoder and a feature decoder are built on the basis of the feature encoder and the feature decoder of the CycleGAN to obtain a light generator network model, and an ADCG network is obtained on the basis of the light generator network model for performing an OPT2SAR task, the light generator network model for performing an SAR2OPT task and a discriminator;
SAR image generation module: and simultaneously inputting the optical ship image in the source domain optical data set and the SAR ship image in the target domain SAR data set obtained by preprocessing into an ADCG network to train the optical ship image and generating a pseudo SAR image of the optical ship image to be converted by utilizing the trained ADCG network.
Further, the specific implementation steps of the preprocessing module are as follows:
s1.1, unifying all the acquired optical ship images in the source domain optical data set to 256 x 256 sizes by using a linear interpolation method;
s1.2, based on the size of each optical ship image in the source domain optical data set, firstly cutting the center of each SAR ship image in the acquired target domain SAR data set to 128 x 128 size, and then amplifying the cut image to 256 x 256 size by using a linear interpolation method.
Further, the dense connection feature converter in the ADCG network building module comprises three DenseBlock blocks connected in sequence, wherein the first DenseBlock block comprises a basic block and two Bottleneck blocks connected in sequence, the basic block comprises a Conv layer, the Bottleneck block comprises a 1x1Conv layer and a 3x3Conv layer connected in sequence, and Concat operation is performed on feature channels before convolution of the 1x1Conv layer and after convolution of the 3x3Conv layer, the DenseBlock represents a dense block, the basic block represents a basic block, and the Bottleneck block represents a bottleneck block;
the second DenseBlock block and the third DenseBlock block respectively comprise three Bottleneck blocks which are sequentially connected;
the light attention mechanism module in the ADCG network construction module comprises a CAM sub-module and a SAM sub-module, wherein the input of the CAM sub-module is the output of the densely connected feature converter, the result of multiplying the output of the SAM sub-module by the input of the SAM sub-module is the input of the feature decoder, the input of the SAM sub-module is the result of multiplying the output of the CAM sub-module by the input of the SAM sub-module by the element, the CAM sub-module represents the channel attention module, and the SAM sub-module represents the space attention module;
the CAM submodule comprises an average pooling layer and a maximum pooling layer, a multi-layer perceptron with a plurality of hidden layers is respectively connected with the average pooling layer and the maximum pooling layer, and element-by-element summation is used for merging output characteristic vectors of results output by the multi-layer perceptron, so that a channel attention diagram is obtained;
the SAM submodule comprises a maximum pooling layer and an average pooling layer which are sequentially connected, a convolution layer for splicing the output of the average pooling layer, and a result output by the convolution layer, namely, a spatial attention diagram is obtained;
in the ADCG network construction module, the expression of the game form of the lightweight generator network model and the discriminator in the ADCG network is as follows:
wherein G represents a lightweight generator network model, D represents a discriminator, i.e., a discriminator, E represents expectations, v-P r (v) Representing real data v and data distribution P thereof r (v)，z～P g (z) represents dummy data z and its specific probability distribution P g (z)，min G max D V (D, G) represents lightweight generationMaximum and minimum betting between the discriminator network model and the discriminator;
the total loss function of the ADCG network is expressed as:
L OPT 2SAR-GAN ＝αL GAN +βL cycle +γL identity
wherein α, β and γ each represent L GAN 、L cycle And L identity Weights of (2);
the OPT2SAR process for the formation of resistant gaming is described as follows:
wherein G is O→S Representing a lightweight generator network model in the process of forming an OPT2SAR mission-resistant game, the OPT2SAR mission representing the conversion of an optical ship image into a pseudo SAR image, D SAR Discriminator for representing a real SAR image and a pseudo SAR image, { x }, x SAR ～X SAR }、{x opt ～X opt Respectively representing the preprocessed source domain optical data set and target domain SAR data set, xSAR and xo p t represents an optical ship image and a SAR ship image from a source domain optical dataset and a target domain SAR dataset, respectively;
the periodic consistency loss function is defined as:
wherein G is S→O (G O→S (x opt ))≈x opt And G O→S (G S→O (x SAR ))≈x SAR ，G S→O A lightweight generator network model in the SAR20PT task resistance game forming process is represented, and the SAR20PT task represents the conversion of an SAR ship image into a pseudo optical image;
the identity loss function is:
a SAR ship identification cross-modal domain migration identification system, comprising an identification module: the SAR ship target recognition method is used for expanding a target domain SAR data set of the obtained pseudo SAR image, and training three classification networks of VGG16, resNet50 and MobileNet based on the expanded target domain SAR data set to recognize the SAR ship target in the SAR ship image to be recognized.
Compared with the prior art, the invention has the advantages that:
1. the invention solves the problems of unbalanced distribution of the data types and insufficient data quantity of the training samples by utilizing the improved generated countermeasure network (namely the ADCG network), and can rapidly extract the characteristics of the ship main body; the improved generation of the anti-network (namely ADCG network) is used in the invention, the network parameter quantity of the densely connected feature converters in the anti-network is greatly reduced, so that the training time of the ADCG network is halved, the feature extraction effect is excellent, the problem of feature redundancy caused by stacking the Residual blocks of the original CycleGAN network is solved, and the light-weight attention mechanism module can inhibit the negative influence of the complex shore-based background and focus the main features of the ship so as to facilitate the rapid and accurate extraction of the features;
2. the characteristics of the pseudo SAR image generated by the ADCG network and the characteristics of the real SAR ship image are extremely mixed, so that the domain gap is remarkably reduced, the pseudo SAR domain and the real SAR domain can not be distinguished almost, and the image conversion task is effectively realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and should not be considered limiting the scope, and that other related drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a general frame diagram of the present invention;
FIG. 2 is a block diagram of a lightweight generator network model in an ADCG network of the invention;
FIG. 3 is a schematic illustration of the data flow and loss function of ADCG in the present invention, wherein input S represents an input SAR image (SAR ship image), input O represents an input optical ship image, and same "represents a data flow and loss function in a lightweight generator network model G S→O Class OPT image obtained by inputting OPT image (optical ship image), same S' represents the light weight generator network model G O→S The SAR-like image obtained by inputting the SAR image, pseudo o 'represents a pseudo OPT image, pseudo S' represents a pseudo SAR image, and recoveredRepresenting SAR images through lightweight generator network model G S→O Obtaining a pseudo OPT image, and the image is subjected to a lightweight generator network model G O→S Obtaining a reconstructed SAR image>Representing an OPT image through a lightweight generator network model G O→S Obtaining a pseudo SAR image, and the image is subjected to a lightweight generator network model G S→O The obtained reconstructed OPT image;
FIG. 4 is a detailed block diagram of a dense connection feature converter of the present invention;
fig. 5 is a detailed block diagram of a lightweight attention mechanism module of the present invention, in which,representing element-by-element additions;
FIG. 6 is an example of an ADCG generation countermeasure network ablation experimental image in the present invention, wherein CycleGAN represents a pseudo SAR image generated by a CycleGAN network, DCM represents a network generated pseudo SAR image after replacing RBs feature converter of CycleGAN with DCM dense connection feature converter; the Att-CG represents a pseudo SAR image generated by a lightweight attention mechanism cyclgan network (lightweight generator network model) to which a lightweight attention mechanism module CBAM is added after an RBs feature converter of cyclgan; ADCG represents a pseudo SAR image generated by the ADCG network presented herein; SAR represents a real SAR image;
FIG. 7 is a t-SNE feature dimension reduction display result of a pseudo SAR domain, a source domain and a target domain in the invention, wherein optical represents a real optical image; pseudoSAR represents a pseudo SAR image; realSAR represents a real SAR image;
FIG. 8 is an example of pseudo SAR domain images generated by ADCG networks and popular generation countermeasure networks in the present invention, wherein U-GAT-IT represents an unsupervised generation attention network based on adaptive layer-instance normalization, dualGAN represents an unsupervised double learning network, and DiscoGAN represents a cross-domain generation countermeasure network;
FIG. 9 is a flow chart comparing accuracy of the classification recognition model before and after expanding the data set in the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1 and fig. 2, the present embodiment discloses a new network ADCG which is more suitable for a ship OPT2SAR cross-modal domain knowledge migration task, namely, provides a SAR ship identification cross-modal domain migration learning method, which includes the following steps:
s1, preprocessing an optical ship image in a source domain optical data set and an SAR ship image in a target domain SAR data set, wherein the specific steps are as follows:
s1.1, unifying all the acquired optical ship images in the source domain optical data set to 256 x 256 sizes by using a linear interpolation method;
s1.2, based on the size of each optical ship image in a source domain optical data set, firstly cutting the center of each SAR ship image in an acquired target domain SAR data set into 128 x 128 sizes, reducing the picture ratio difference of ship main bodies in the source domain optical data set and the target domain SAR data set image, and amplifying the cut image to 256 x 256 sizes by using a linear interpolation method.
S2, keeping the original feature encoder and decoder of the CycleGAN unchanged, constructing a densely connected feature converter to replace the CycleGAN feature converter formed by simply stacking 9 residual blocks, generating a lightweight generator network model, reducing feature redundancy and shortening network training time; the method comprises the steps that a dense connection feature converter and a light attention mechanism module which are connected with a feature encoder and a feature decoder are built on the basis of the feature encoder and the feature decoder of the CycleGAN, a light generator network model is obtained, and an ADCG network is obtained on the basis of a light generator network model which performs an OPT2SAR task, a light generator network model which performs an SAR2OPT task and a discriminator;
the dense connection feature converter comprises three DenseBlock blocks which are sequentially connected, wherein the first DenseBlock block comprises a basic block and two BottenceckBlock blocks which are sequentially connected, the basic block comprises a Conv layer, the BottenceckBlock comprises a 1x1Conv layer and a 3x3Conv layer which are sequentially connected, and Concat operation for splicing feature channels before the convolution of the 1x1Conv layer and after the convolution of the 3x3Conv layer is performed, the DenseBlock represents a dense block, the basic block represents a basic block, and the BottenceckBlock represents a bottleneck block;
the second DenseBlock block and the third DenseBlock block respectively comprise three Bottleneck blocks connected in sequence.
As shown in fig. 4, the DCM (dense connectivity feature converter) has three DenseBlock blocks, each composed of three BottendeckBlock inside, wherein the first DB (DenseBlock) block is slightly different from the last two DB blocks, the first DB block is composed of one Basicblock and two BottendeckBlock blocks, and the Basicblock functions to reduce the number of feature channels of 256 to 128 by using one Conv layer; the BottleneckBlock functions to convolve the number of input characteristic channels to 16 by using a 1x1Conv layer and a 3x3Conv layer, and splice the characteristic channels before and after convolution by using a Concat operation; the final profile can keep the 256 number of feature channels unchanged after passing through 3 DB blocks. The densely connected feature converter greatly reduces the number of feature channels and the network calculation amount while guaranteeing the feature extraction quality, reduces the number of feature graphs, but increases the reuse rate of the feature graphs to reduce network parameters, solves the problem of feature redundancy caused by an original CycleGAN residual block generator, and greatly improves the training speed of a network model.
The light attention mechanism module comprises a CAM sub-module and a SAM sub-module, wherein the input of the CAM sub-module is the output of the intensive connection feature converter, the result of multiplying the output of the SAM sub-module by elements with the input of the CAM sub-module is the input of the feature decoder, the input of the SAM sub-module is the result of multiplying the output of the CAM sub-module by elements with the input of the CAM sub-module, the CAM sub-module represents the channel attention module, and the SAM sub-module represents the space attention module; the CAM submodule comprises an average pooling layer and a maximum pooling layer, a multi-layer perceptron with a plurality of hidden layers is respectively connected with the average pooling layer and the maximum pooling layer, and element-by-element summation is used for merging output characteristic vectors of results output by the multi-layer perceptron, so that a channel attention diagram is obtained; the SAM submodule comprises a maximum pooling layer and an average pooling layer which are sequentially connected, a convolution layer which is used for splicing the output of the average pooling layer, and a result output by the convolution layer, namely the spatial attention diagram is obtained.
The light attention mechanism module helps the dense connection feature converter to eliminate complex shore-based background interference and quickly position the main features of the image ship. A detailed block diagram of the lightweight attention mechanism module of the present invention is shown in fig. 5. CBAM (lightweight attention mechanism module) contains two sub-modules CAM (Channel Attention Module) and SAM (Spartial Attention Module), given an intermediate feature mapAs input, the CBAM module sequentially performs channel and spatial attention calculations to generate a 1D channel attention map ++>And a 2D spatial attentionForce striving->The CBAM module can adaptively refine the intermediate feature map, so that the computational power is ensured and the parameters are saved. The overall process can be summarized as follows:
wherein,,representing element-by-element multiplication;
the CAM module uses the average pooling and the maximum pooling operation to aggregate the channel information of the feature map so as to improve the representation capability of the network. The generated average pooling feature and maximum pooling feature are forwarded to a shared network consisting of multi-layer perceptrons (MLPs) with multiple hidden layers to produce a channel attention mapFinally, merging the output eigenvectors by element-by-element summation to obtain the channel attention map M c And multiplying the intermediate feature map F element by element to obtain an output feature F'.
The spatial relationship of features is used in the SAM sub-module to generate a spatial attention map. The module first applies average pooling and maximum pooling along the channel axis to effectively highlight the information region, then concatenates the feature descriptions of the two, and applies a convolution layer to generate a spatial attention mapAnd encoding the emphasized or suppressed region to obtain a spatial attention map M s And multiplying the characteristic F 'element by element to obtain an output characteristic F'.
The CBAM module adaptively refines the intermediate feature map extracted by the feature converter by using the channel and the position attention mechanism, so that the function of positioning the main feature of the ship is realized, the number of light network parameters is maintained, and the burden of greatly increasing the training time of the network model is avoided.
S3, inputting the optical ship image in the source domain optical data set and the SAR ship image in the target domain SAR data set obtained through preprocessing into an ADCG network at the same time to train the optical ship image and generating a pseudo SAR image of the optical ship image to be converted by utilizing the trained ADCG network.
The training step of the ADCG network comprises the following logic:
the preprocessed optical ship image in the source domain optical data set and SAR ship image in the target domain SAR data set are input into G at the same time O→S Training a lightweight generator network model of an OPT2SAR process while inputting G S→O The lightweight generator network model of the SAR2OPT process is trained, and during the training process, the two lightweight generator network models form a closed loop, as shown in fig. 3. The antagonism between the corresponding lightweight generator network model and the discriminant during training indirectly improves the performance of the generator. The specific process after the image input generator comprises the following steps: downsampling by a feature encoder, feature conversion by a DCM module, attention profile construction by a CBAM module, pixel reconstruction by a feature decoder.
The generation step of the pseudo SAR image comprises the following steps:
using only trained G O→S And a generator for inputting the OPT image (optical ship image) to be converted and generating a pseudo SAR image (pseudo SAR ship image). The specific process is as follows: the OPT image input feature encoder downsamples-feature conversion by densely connected feature converters-attention profile construction by CBAM-pixel reconstruction by feature decoder.
And obtaining an SAR data set of the pseudo SAR image expansion target domain, and training three classification networks VGG16, resNet50 and MobileNet based on the SAR data set of the expanded target domain to perform SAR ship target identification in the SAR ship image to be identified.
In this example, to illustrate the superiority of generating an anti-network ADCG network we have proposed, an ablation experiment was first performed, with the experimental results shown in fig. 6, for example. The improved densely connected feature converter has the advantages that the network parameter number is greatly reduced, the training time is halved, the feature extraction effect is excellent, the improved densely connected feature converter solves the problem of feature redundancy caused by stacking the Residual blocks in the original CycleGAN network, and the feature extraction effectiveness of the densely connected feature converter is proved. The CycleGAN and DenseBlock network parameters added with the lightweight attention mechanism CBAM module are little increased, training time is hardly increased compared with that of a network model without the CBAM module, and experimental results also show that the CBAM module has obvious lifting effect on the network model, can inhibit the negative influence of a complex shore-based background and focus the main body characteristics of a ship.
In order to further verify the effectiveness of the method on the cross-modal domain pixel level feature migration task of the ship OPT2SAR, the optical domain training set, the SAR domain training set and the pseudo SAR domain testing set feature distribution are displayed in a 2D image dimension reduction mode by using a t-SNE method, and a feature dimension reduction display result is shown in figure 7. It can be clearly seen after mapping each corresponding feature in the three domains into the 2D space, although there are a few dark dots representing the left darkest feature of real_sar_img intersecting with slightly lighter dark dots representing the right side feature of optical_img, there is still a wide distribution gap between the Optical domain and SAR domain images, which illustrates that the task of OPT2SAR cross-modal domain knowledge migration is indeed very difficult. However, the proposed method for generating the countermeasure converts the optical domain image into the pseudo SAR domain image in a cross-modal mode, the generated pseudo SAR domain ship image features and the real SAR domain ship image features are extremely mixed, the domain gap is remarkably reduced, the pseudo SAR domain and the real SAR domain can not be distinguished almost, and the superior performance of the proposed method in the ship OPT2SAR image conversion task is also shown.
Next, we have conducted a comparison experiment between the proposed model and several popular unsupervised, unpaired image style conversion methods, and the experimental results are shown in fig. 8, to verify that the proposed model achieves superior ship OPT2SAR image style conversion performance. Comparing all popular generation countermeasure networks with the ADCG network proposed by us, the network proposed by us realizes the highest image conversion quality with lower model training time, and the CBAM module realizes the effect of inhibiting complex shore-based background, thereby perfectly fitting the ship OPT2SAR image conversion task.
Finally, in order to verify the rationality and effectiveness of a pseudo SAR domain generated by a ship OPT2SAR cross-modal domain pixel level feature migration task on solving the problems of scarcity and unbalanced category of effective marking samples of an SAR ship ATR deep learning network model, the capability of improving the identification precision of a popular ship classification network by the pseudo SAR image is tested. A flow chart for comparing accuracy of the classification recognition model before and after expanding the data set is shown in fig. 9. The three popular classification networks achieve the recognition precision improvement of 0.06 on average, and prove the rationality, the effectiveness and the application value of the pseudo SAR domain generated by the cross-modal domain pixel level feature migration task of the OPT2SAR of the ship to the problem of effectively marking sample scarcity and class imbalance of the deep learning network model of the ATR of the ship.
The invention discloses a new network ADCG which is more suitable for a ship OPT2SAR cross-modal domain knowledge migration task based on a well-known non-supervision and non-pairing image conversion network CycleGAN. And constructing a dense connection module generator, and solving the problems of generator feature redundancy and time-consuming network model training of a residual block generator of an original CycleGAN network in the task. A light attention mechanism module is constructed to help the generator solve the problem that the main body characteristics of the ship can not be positioned under the condition of increasing network parameters in a very small amount. And finally, training the ADCG network according to preset training parameters and a loss function, generating a pseudo SAR image by using the trained ADCG network, and constructing a pseudo SAR domain driven SAR ship target recognition task. The invention can show the most excellent performance in the cross-modal domain pixel level knowledge migration task of the ship OPT2SAR by using the lowest FID value and the KID value, and the generated pseudo SAR domain has great rationality and application value for solving the problems of scarce effective marking samples and unbalanced categories of the SAR ship ATR deep learning network model.

Claims (10)

1. The SAR ship identification cross-modal domain migration learning method is characterized by comprising the following steps of:
s1, preprocessing an optical ship image in a source domain optical data set and an SAR ship image in a target domain SAR data set;
s2, a dense connection feature converter and a light-weight attention mechanism module which are connected with the feature encoder and the feature decoder are constructed based on the feature encoder and the feature decoder of the CycleGAN to obtain a light-weight generator network model, and an ADCG network is obtained based on the light-weight generator network model for performing an OPT2SAR task, the light-weight generator network model for performing an SAR2OPT task and the discriminator;
s3, inputting the optical ship image in the source domain optical data set and the SAR ship image in the target domain SAR data set obtained through preprocessing into an ADCG network at the same time to train the optical ship image and generating a pseudo SAR image of the optical ship image to be converted by utilizing the trained ADCG network.
2. The SAR ship identification cross-modal domain transfer learning method according to claim 1, wherein the specific steps of step S1 are as follows:
s1.1, unifying all the acquired optical ship images in the source domain optical data set to 256 x 256 sizes by using a linear interpolation method;
s1.2, based on the size of each optical ship image in the source domain optical data set, firstly cutting the center of each SAR ship image in the acquired target domain SAR data set to 128 x 128 size, and then amplifying the cut image to 256 x 256 size by using a linear interpolation method.
3. The SAR ship identification cross-modal domain transfer learning method of claim 2, wherein the method comprises the steps of: the dense connection feature converter comprises three DenseBlock blocks which are sequentially connected, wherein the first DenseBlock block comprises a basic block and two BottenceckBlock blocks which are sequentially connected, the basic block comprises a Conv layer, the BottenceckBlock comprises a 1x1Conv layer and a 3x3Conv layer which are sequentially connected, and Concat operation for splicing feature channels before the convolution of the 1x1Conv layer and after the convolution of the 3x3Conv layer is performed, the DenseBlock represents a dense block, the basic block represents a basic block, and the BottenceckBlock represents a bottleneck block;
the second DenseBlock block and the third DenseBlock block respectively comprise three Bottleneck blocks connected in sequence.
4. The method for cross-modal domain transfer learning of SAR ship identification as claimed in claim 3, wherein the light attention mechanism module comprises a CAM sub-module and a SAM sub-module, the input of the CAM sub-module is the output of a densely connected feature converter, the result of multiplying the output of the SAM sub-module by the input of the SAM sub-module is the input of a feature decoder, the input of the SAM sub-module is the result of multiplying the output of the CAM sub-module by the input of the SAM sub-module by the element, the CAM sub-module represents a channel attention module, and the SAM sub-module represents a space attention module;
the CAM submodule comprises an average pooling layer and a maximum pooling layer, a multi-layer perceptron with a plurality of hidden layers is respectively connected with the average pooling layer and the maximum pooling layer, and element-by-element summation is used for merging output characteristic vectors of results output by the multi-layer perceptron, so that a channel attention diagram is obtained;
the SAM submodule comprises a maximum pooling layer and an average pooling layer which are sequentially connected, a convolution layer which is used for splicing the output of the average pooling layer, and a result output by the convolution layer, namely the spatial attention diagram is obtained.
5. The SAR ship identification cross-modal domain transfer learning method of claim 4, wherein the lightweight generator network model and discriminator game form expression in the ADCG network is:
wherein G representsLightweight generator network model, D represents a discriminator, i.e., discriminator, E represents expectations, v-P r (v) Representing real data v and data distribution P thereof r (v)，z～P g (z) represents dummy data z and its specific probability distribution P g (z)，min G max D V (D, G) represents the maximum minimum game between the lightweight generator network model and the discriminator;
the total loss function of the ADCG network is expressed as:
L OPT2SAR-GAN ＝αL GAN +βL cycle +γL identity
wherein α, β and γ each represent L GAN 、L cycle And L identity Weights of (2);
the OPT2SAR process for the formation of resistant gaming is described as follows:
wherein G is O→S Representing a lightweight generator network model in the process of forming an OPT2SAR mission-resistant game, the OPT2SAR mission representing the conversion of an optical ship image into a pseudo SAR image, D SAR Discriminator for representing a real SAR image and a pseudo SAR image, { x }, x SAR ～X SAR }、{X opt ～X opt Respectively representing a source domain optical data set and a target domain SAR data set obtained after preprocessing, x SAR And x opt Representing an optical ship image and a SAR ship image from a source domain optical dataset and a target domain SAR dataset, respectively;
the periodic consistency loss function is defined as:
wherein G is S→O (G O→S (x opt ))≈x opt And G O→S (G S→O (x SAR ))≈x SAR ，G S→O Representing SAR2OPTThe SAR2OPT task represents converting SAR ship images into pseudo-optical images;
the identity loss function is:
6. a method for identifying the migration and the identification of a SAR ship across modal domains is characterized by comprising the following steps: the method comprises the steps of expanding a target domain SAR data set of a pseudo SAR image obtained according to any one of claims 1-5, and training three classification networks of VGG16, resNet50 and MobileNet based on the expanded target domain SAR data set to perform SAR ship target recognition in a to-be-recognized SAR ship image.
7. A SAR ship identification cross-modal domain transfer learning system, comprising:
and a pretreatment module: preprocessing an optical ship image in a source domain optical data set and an SAR ship image in a target domain SAR data set;
ADCG network construction module: the method comprises the steps that a dense connection feature converter and a light attention mechanism module which are connected with a feature encoder and a feature decoder are built on the basis of the feature encoder and the feature decoder of the CycleGAN to obtain a light generator network model, and an ADCG network is obtained on the basis of the light generator network model for performing an OPT2SAR task, the light generator network model for performing an SAR2OPT task and a discriminator;
SAR image generation module: and simultaneously inputting the optical ship image in the source domain optical data set and the SAR ship image in the target domain SAR data set obtained by preprocessing into an ADCG network to train the optical ship image and generating a pseudo SAR image of the optical ship image to be converted by utilizing the trained ADCG network.
8. The SAR ship identification cross-modal domain transfer learning system of claim 7, wherein the preprocessing module comprises the following steps:
s1.1, unifying all the acquired optical ship images in the source domain optical data set to 256 x 256 sizes by using a linear interpolation method;
s1.2, based on the size of each optical ship image in the source domain optical data set, firstly cutting the center of each SAR ship image in the acquired target domain SAR data set to 128 x 128 size, and then amplifying the cut image to 256 x 256 size by using a linear interpolation method.
9. The SAR ship identification cross-modal domain transfer learning system of claim 8, wherein: the dense connection feature converter in the ADCG network construction module comprises three DenseBlock blocks which are sequentially connected, wherein the first DenseBlock block comprises a Basicblock and two BottleneckBlock blocks which are sequentially connected, the Basicblock comprises a Conv layer, the BottleneckBlock comprises a 1x1Conv layer and a 3x3Conv layer which are sequentially connected, and Concat operation for splicing feature channels before convolution of the 1x1Conv layer and after convolution of the 3x3Conv layer is performed, the DenseBlock represents a dense block, the Basicblock represents a basic block, and the Bottleneckblock represents a bottleneck block;
the second DenseBlock block and the third DenseBlock block respectively comprise three Bottleneck blocks which are sequentially connected;
the light attention mechanism module in the ADCG network construction module comprises a CAM sub-module and a SAM sub-module, wherein the input of the CAM sub-module is the output of the densely connected feature converter, the result of multiplying the output of the SAM sub-module by the input of the SAM sub-module is the input of the feature decoder, the input of the SAM sub-module is the result of multiplying the output of the CAM sub-module by the input of the SAM sub-module by the element, the CAM sub-module represents the channel attention module, and the SAM sub-module represents the space attention module;
the CAM submodule comprises an average pooling layer and a maximum pooling layer, a multi-layer perceptron with a plurality of hidden layers is respectively connected with the average pooling layer and the maximum pooling layer, and element-by-element summation is used for merging output characteristic vectors of results output by the multi-layer perceptron, so that a channel attention diagram is obtained;
the SAM submodule comprises a maximum pooling layer and an average pooling layer which are sequentially connected, a convolution layer for splicing the output of the average pooling layer, and a result output by the convolution layer, namely, a spatial attention diagram is obtained;
in the ADCG network construction module, the expression of the game form of the lightweight generator network model and the discriminator in the ADCG network is as follows:
wherein G represents a lightweight generator network model, D represents a discriminator, i.e., a discriminator, E represents expectations, v-P r (v) Representing real data v and data distribution P thereof r (v)，z～P g (z) represents dummy data z and its specific probability distribution P g (z)，min G max D V (D, G) represents the maximum minimum game between the lightweight generator network model and the discriminator;
the total loss function of the ADCG network is expressed as:
L OPT2SAR-GAN ＝αL GAN +βL cycle +γL identity
wherein α, β and γ each represent L GAN 、L cycle And L identity Weights of (2);
the OPT2SAR process for the formation of resistant gaming is described as follows:
wherein G is O→S Representing a lightweight generator network model in the process of forming an OPT2SAR mission-resistant game, the OPT2SAR mission representing the conversion of an optical ship image into a pseudo SAR image, D SAR Discriminator for representing a real SAR image and a pseudo SAR image, { x }, x SAR ～X SAR }、{x opt ～X opt Respectively representing a source domain optical data set and a target domain SAR data set obtained after preprocessing, x SAR And x opt Representing an optical ship image and a SAR ship image from a source domain optical dataset and a target domain SAR dataset, respectively;
the periodic consistency loss function is defined as:
wherein G is S→O (G O→S (x opt ))≈x opt And G O→S (G S→O (x SAR ))≈x SAR ，G S→O A lightweight generator network model in the SAR2OPT task opposite game forming process is represented, and SAR2OPT task represents that SAR ship images are converted into pseudo-optical images;
the identity loss function is:
10. a SAR ship identification cross-modal domain migration identification system, comprising an identification module: the method is used for expanding a target domain SAR data set of the pseudo SAR image obtained according to any one of claims 7-9, and training three classification networks of VGG16, resNet50 and Mobi leNet based on the expanded target domain SAR data set to perform SAR ship target recognition in the SAR ship image to be recognized.
CN202310750641.8A 2023-06-25 2023-06-25 SAR ship identification cross-modal domain migration learning and identification method and system Pending CN116912680A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310750641.8A CN116912680A (en) 2023-06-25 2023-06-25 SAR ship identification cross-modal domain migration learning and identification method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310750641.8A CN116912680A (en) 2023-06-25 2023-06-25 SAR ship identification cross-modal domain migration learning and identification method and system

Publications (1)

Publication Number Publication Date
CN116912680A true CN116912680A (en) 2023-10-20

Family

ID=88355445

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310750641.8A Pending CN116912680A (en) 2023-06-25 2023-06-25 SAR ship identification cross-modal domain migration learning and identification method and system

Country Status (1)

Country Link
CN (1) CN116912680A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117115170A (en) * 2023-10-25 2023-11-24 安徽大学 Self-adaptive SAR ship detection method and system in unsupervised domain
CN117576236A (en) * 2023-11-13 2024-02-20 宁波大学 Missing optical image reconstruction method combining multi-phase SAR and optical information

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111461169A (en) * 2020-03-04 2020-07-28 浙江工商大学 Pedestrian attribute identification method based on forward and reverse convolution and multilayer branch depth network
CN112132104A (en) * 2020-10-09 2020-12-25 哈尔滨工业大学 ISAR ship target image domain enhancement identification method based on loop generation countermeasure network
CN113838107A (en) * 2021-09-23 2021-12-24 哈尔滨工程大学 Dense connection-based heterogeneous image automatic registration method
CN114519678A (en) * 2022-02-17 2022-05-20 烟台大学 Scanning transmission image recovery method, device and storage medium
CN115376066A (en) * 2022-08-17 2022-11-22 东南大学 Airport scene target detection multi-weather data enhancement method based on improved cycleGAN
CN115861599A (en) * 2022-12-14 2023-03-28 西安镭映光电科技有限公司 Infrared weak and small target detection method based on infrared small sample amplification and YOLOv5
CN115909002A (en) * 2022-10-10 2023-04-04 南京莱斯电子设备有限公司 Image translation method based on contrast learning
CN115908602A (en) * 2022-09-29 2023-04-04 西北大学 Style migration method for converting landscape photos into Chinese landscape paintings

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111461169A (en) * 2020-03-04 2020-07-28 浙江工商大学 Pedestrian attribute identification method based on forward and reverse convolution and multilayer branch depth network
CN112132104A (en) * 2020-10-09 2020-12-25 哈尔滨工业大学 ISAR ship target image domain enhancement identification method based on loop generation countermeasure network
CN113838107A (en) * 2021-09-23 2021-12-24 哈尔滨工程大学 Dense connection-based heterogeneous image automatic registration method
CN114519678A (en) * 2022-02-17 2022-05-20 烟台大学 Scanning transmission image recovery method, device and storage medium
CN115376066A (en) * 2022-08-17 2022-11-22 东南大学 Airport scene target detection multi-weather data enhancement method based on improved cycleGAN
CN115908602A (en) * 2022-09-29 2023-04-04 西北大学 Style migration method for converting landscape photos into Chinese landscape paintings
CN115909002A (en) * 2022-10-10 2023-04-04 南京莱斯电子设备有限公司 Image translation method based on contrast learning
CN115861599A (en) * 2022-12-14 2023-03-28 西安镭映光电科技有限公司 Infrared weak and small target detection method based on infrared small sample amplification and YOLOv5

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
YU SHI等: "Unsupervised Domain Adaptation Based on Progressive Transfer for Ship Detection:From Optical to SAR Images", 《IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING》, 22 June 2022 (2022-06-22), pages 1 - 17, XP011913500, DOI: 10.1109/TGRS.2022.3185298 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117115170A (en) * 2023-10-25 2023-11-24 安徽大学 Self-adaptive SAR ship detection method and system in unsupervised domain
CN117115170B (en) * 2023-10-25 2024-01-12 安徽大学 Self-adaptive SAR ship detection method and system in unsupervised domain
CN117576236A (en) * 2023-11-13 2024-02-20 宁波大学 Missing optical image reconstruction method combining multi-phase SAR and optical information

Similar Documents

Publication Publication Date Title
CN112465111B (en) Three-dimensional voxel image segmentation method based on knowledge distillation and countermeasure training
CN116912680A (en) SAR ship identification cross-modal domain migration learning and identification method and system
CN114202696A (en) SAR target detection method and device based on context vision and storage medium
CN110363068B (en) High-resolution pedestrian image generation method based on multiscale circulation generation type countermeasure network
CN106355151A (en) Recognition method, based on deep belief network, of three-dimensional SAR images
CN112967178B (en) Image conversion method, device, equipment and storage medium
CN112330724B (en) Integrated attention enhancement-based unsupervised multi-modal image registration method
CN113283444B (en) Heterogeneous image migration method based on generation countermeasure network
CN113379601A (en) Real world image super-resolution method and system based on degradation variational self-encoder
CN111210382A (en) Image processing method, image processing device, computer equipment and storage medium
CN112950780A (en) Intelligent network map generation method and system based on remote sensing image
CN116542996B (en) Spine CT image segmentation method based on feature fusion and difference
CN113256520A (en) Domain-adaptive underwater image enhancement method
CN114612769A (en) Integrated sensing infrared imaging ship detection method integrated with local structure information
CN109977968A (en) A kind of SAR change detecting method of deep learning classification and predicting
CN117058367A (en) Semantic segmentation method and device for high-resolution remote sensing image building
CN112560624A (en) High-resolution remote sensing image semantic segmentation method based on model depth integration
CN112488963A (en) Method for enhancing crop disease data
CN114639002A (en) Infrared and visible light image fusion method based on multi-mode characteristics
CN114299130A (en) Underwater binocular depth estimation method based on unsupervised adaptive network
CN104036242A (en) Object recognition method based on convolutional restricted Boltzmann machine combining Centering Trick
CN116664435A (en) Face restoration method based on multi-scale face analysis map integration
CN115578511A (en) Semi-supervised single-view 3D object reconstruction method
CN112330562B (en) Heterogeneous remote sensing image transformation method and system
CN118070227A (en) Multi-mode data processing method enhanced by large language model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination