WO2023050746A1 - 增强舰船目标检测sar图像数据的方法 - Google Patents

增强舰船目标检测sar图像数据的方法 Download PDF

Info

Publication number
WO2023050746A1
WO2023050746A1 PCT/CN2022/083836 CN2022083836W WO2023050746A1 WO 2023050746 A1 WO2023050746 A1 WO 2023050746A1 CN 2022083836 W CN2022083836 W CN 2022083836W WO 2023050746 A1 WO2023050746 A1 WO 2023050746A1
Authority
WO
WIPO (PCT)
Prior art keywords
ship
sar image
image
sar
data
Prior art date
Application number
PCT/CN2022/083836
Other languages
English (en)
French (fr)
Inventor
代翔
王侃
崔莹
潘磊
高翔
廖泓舟
Original Assignee
西南电子技术研究所（中国电子科技集团公司第十研究所）
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西南电子技术研究所（中国电子科技集团公司第十研究所） filed Critical 西南电子技术研究所（中国电子科技集团公司第十研究所）
Publication of WO2023050746A1 publication Critical patent/WO2023050746A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G06T2207/10044—Radar image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • the invention relates to data enhancement in the field of artificial intelligence, in particular to an image data intelligent enhancement technology based on a generative confrontation network, in particular to a SAR image data enhancement method for ship target detection.
  • Synthetic Aperture Radar has the characteristics of all-day, all-weather, high resolution, and wide surveying swath.
  • Strong penetrating active microwave remote sensing technology is an indispensable and important means for ocean information acquisition and monitoring. Since the SAR image reflects the electromagnetic scattering characteristics of the target, the homogeneous clutter and artificial clutter contained in it reduce the accuracy of traditional target detection methods.
  • the azimuth ambiguity caused by the ship target itself is also a difficult problem in ship target detection. The azimuth ambiguity of the ship is mainly caused by the single or double scattering of the ship.
  • the data used in traditional ship target detection is single-channel SAR data.
  • CFAR and its various improved algorithms are most widely used in the field of SAR image ship detection. It detects ship targets by modeling the statistical distribution of background clutter. Due to the coherent imaging characteristics of the SAR system and the complexity of the sea surface conditions, the clutter distribution in the SAR image has strong time-varying, non-stationary, and non-Gaussian characteristics, which makes the CFAR detection algorithm have a high false alarm rate. Moreover, the CFAR detection algorithm largely depends on the accurate estimation of the clutter model, false alarm rate, target window, protection window and background window, which makes the detection performance have great uncertainty in practical applications.
  • the identified data set needs to collect a large number of samples of various ship targets, which is difficult.
  • the data set SSDD is obtained by downloading public SAR images from the Internet, cutting the target area into a size of about 500 ⁇ 500 pixels, and manually marking the target position of the ship.
  • SSDD is the first publicly available data set dedicated to ship target detection in SAR images at home and abroad. It can be used to train and test detection algorithms, allowing researchers to compare algorithm performance under the same conditions. For each ship, the detection algorithm predicts the bounding box of the ship object and gives a confidence that it is a ship object.
  • SSDD is produced by referring to the production process of the PASCAL VOC dataset. This is because PASCAL VOC is widely used in the field of target detection, and the data format is more standardized.
  • GAN Generative Adversarial Networks
  • Satellite remote sensing technology is an important means of surveillance and monitoring of ships at sea, and plays an important role in ship detection, classification, recognition and motion feature extraction. Meet the needs of related applications.
  • the purpose of this disclosure is to solve the problems of low quality of generated data, no labels, and poor practicability in existing SAR image data enhancement methods, and to provide a stable training, robust model, diversity and practicability, which can improve the generation of SAR Image data quality, SAR image data enhancement method for ship target detection.
  • the ship position in image form is used as the constraint condition c of SAR image enhancement
  • the constraint condition c and the hidden variable z are used as the input of the conditional generation confrontation network generator based on the ship position information
  • the hidden variable After z passes through two fully connected layers, a high-dimensional feature vector is obtained, and it is reconstructed into a hidden variable feature map.
  • the constraint c obtains a conditional feature map through a convolutional layer.
  • the hidden variable feature map and the conditional feature map are cascaded and input to At least 4 layers of transposed convolutional layers are obtained to obtain a comprehensive feature map, and the feature resolution is increased by layer-by-layer upsampling to generate a new SAR ship image, and the target frame as the constraint c is correspondingly converted into the label of the generated SAR data;
  • the discriminator combines the ship position information as a constraint to form a data-label pair as the input of the discriminator network, and the discriminator extracts the features of the data-label pair through the convolutional layer , to judge the authenticity of the generated SAR image, and at the same time judge the matching degree of the generated image/real image with the corresponding constraint conditions, through the confrontation learning of the generator and the discriminator, the generated SAR image is closer to the real SAR image; finally, the constructed
  • the conditional generation confrontation network model is cascaded with the target detection network, and the quality of the generated SAR image data
  • This disclosure is oriented towards ship target detection, centering on the ship position, taking the ship position in the form of an image as the constraint condition c of SAR image enhancement, and the constraint condition c and the latent variable z as the condition generation based on the ship position information
  • the input of the confrontation network generator, the latent variable z passes through two fully connected layers to obtain a high-dimensional feature vector, and reconstructs it into a hidden variable feature map
  • the constraint condition c obtains a conditional feature map through a convolutional layer
  • the hidden variable feature map After cascading with the conditional feature map, it is input to at least 4 layers of transposed convolutional layers to obtain a comprehensive feature map, upsampling layer by layer to improve the feature resolution, generate a new SAR ship image, and convert the target frame corresponding to the constraint condition c label of the generated SAR data; for the input real SAR image and the generated SAR image, the discriminator combines the position information of the ship as a constraint to form a data-l
  • This conditional generative adversarial network based on ship position information can better measure the distance between real data and generated data by using the optimal transmission distance as a measurement function, and correctly guide the gradient descent direction of the model, so that the training of the model is stable.
  • the model is robust. It can avoid that the objective function in the existing generative confrontation network method is difficult to effectively guide the gradient descent, resulting in unstable training or even mode collapse.
  • the hidden variable feature map and the conditional feature map are cascaded and input to at least 4 layers of transposed convolutional layers to obtain a comprehensive feature map, and the feature resolution is improved by layer-by-layer upsampling to generate a new SAR ship image.
  • the ship position information-assisted conditional generation confrontation network of the present disclosure uses the ship position information as the constraint condition of the generator and the label of the generated image. After the network training is completed, it can directly generate new SAR image data with labels to overcome the It overcomes the defect that the SAR images generated by traditional GAN have no labels and need to be manually marked before they can be used as training data.
  • This disclosure cascades the constructed conditional generation adversarial network model based on ship position information with the target detection network, evaluates the quality of SAR image data generated by ship target detection results, and feeds back the detection results to condition generation
  • the adversarial network continuously optimizes the generator and encourages it to generate higher-quality SAR image new data, and enhances the SAR image data through the cooperative learning of the adversarial network and the target detection network. It has diversity and practicability, and can improve the quality of generated SAR image data.
  • Fig. 1 is a schematic diagram of the present invention's conditional generation confrontation network based on ship position information
  • Fig. 2 is a schematic diagram of the collaborative learning framework of conditional generative adversarial network and target detection constructed by the present invention.
  • the ship position in image form is used as the constraint condition c of SAR image enhancement
  • the constraint condition c and the hidden variable z are used as the input of the conditional generation confrontation network generator based on the ship position information
  • the hidden variable After z passes through two fully connected layers, a high-dimensional feature vector is obtained, and it is reconstructed into a hidden variable feature map.
  • the constraint c obtains a conditional feature map through a convolutional layer.
  • the hidden variable feature map and the conditional feature map are cascaded and input to At least 4 layers of transposed convolutional layers are obtained to obtain a comprehensive feature map, and the feature resolution is increased by layer-by-layer upsampling to generate a new SAR ship image, and the target frame as the constraint c is correspondingly converted into the label of the generated SAR data;
  • the discriminator combines the ship position information as a constraint to form a data-label pair as the input of the discriminator network, and the discriminator extracts the features of the data-label pair through the convolutional layer , to judge the authenticity of the generated SAR image, and at the same time judge the matching degree of the generated image/real image with the corresponding constraint conditions, through the confrontation learning of the generator and the discriminator, the generated SAR image is closer to the real SAR image; finally, the constructed
  • the conditional generation confrontation network model is cascaded with the target detection network, and the quality of the generated SAR image data
  • the conditional generation confrontation network based on ship position information includes two parts: generator and discriminator.
  • the generator takes hidden variable z and constraint condition c as input.
  • Hidden variable z passes through two fully connected layers to obtain high-dimensional feature vectors. And reconstruct it into a 4 ⁇ 4 ⁇ 256 hidden variable feature map;
  • the constraint condition c obtains a 4 ⁇ 6 ⁇ 256 conditional feature map through a 4-layer convolutional layer with a step size of 2; after cascading the two feature maps Input to 4 layers of transposed convolutional layers, upsampling layer by layer to improve feature resolution, generate a new SAR ship image with a size of 64 ⁇ 64 pixels, and in order to effectively use feature information, transpose convolutional layers in every two layers Add a residual connection between them.
  • the discriminator introduces the position information of the ship through the image-label pair, and cascades the generated image/real image with the corresponding constraints to form a 64 ⁇ 64 ⁇ 26 network input;
  • the discriminator network structure consists of 4 stacked convolutions Layer, two layers of fully connected layer and softmax layer, the convolutional layer extracts the features of the data-label pair, the fully connected layer converts the feature into a scalar, and the softmax layer is a linear classifier, which can identify the authenticity of the generated SAR image and judge The degree of matching between the generated image/real image and the corresponding constraint conditions, and then realize the feedback adjustment of the generator's generated results, and improve the generation quality of the model.
  • the conditional generative adversarial network based on ship position information introduces the optimal transmission distance to construct the generator objective function: and the objective function of the discriminator:
  • E[ ⁇ ] represents the expected value of the distribution function
  • x ⁇ P r represents that x obeys the probability distribution P r
  • c) is the non-matching constraint of the real image
  • constraint c is the probability of the real image
  • constraint Down is the probability of generating an image.
  • the conditional generative adversarial network based on ship position information is connected with the target detection network to construct a feedback self-optimized SAR image enhancement model, which is mainly composed of constraint conditions, conditional generative adversarial network based on ship position information, and enhanced SAR image data quality assessment. Partial composition. Taking the real SAR image as input, the position constraint of the ship target is generated through the constraint building module, and the conditional generation confrontation network based on the ship position information is input to generate the SAR image, and the quality of the generated SAR image is evaluated by the YOLO V3 target detection network model. The ship detection results are fed back to the conditional generation adversarial network, which motivates the generator to generate higher-quality SAR images. Finally, the adversarial generation network and the target detection network achieve collaborative learning, forming an orderly autonomous iterative cycle optimization mechanism.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Radar Systems Or Details Thereof (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开的一种增强舰船目标检测SAR图像数据的方法，训练稳定、模式稳健。可以通过下述技术方案实现：以舰船位置为中心，将图像形式的舰船位置作为SAR图像增强的约束条件，通过两个全连接层后将得到的高维特征向量重构为条件特征图，条件特征图与隐变量特征图级联后输入到转置卷积层得到综合特征图，逐层上采样提高特征分辨率，生成新的SAR舰船图像，将目标框对应转换成所生成SAR图像的标签，构建数据-标签对；判别器通过卷积层提取数据-标签对的特征，判别生成SAR图像的真假和图像与标签的匹配程度，通过生成器与判别器的对抗，激励生成器生成更高质量的SAR图像新数据；最后通过对抗网络与目标检测网络的协同学习，增强SAR图像数据。

Description

增强舰船目标检测SAR图像数据的方法
本申请要求于2021年09月30日提交中国专利局、申请号为202111159144.8、申请名称“增强舰船目标检测SAR图像数据的方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及人工智能领域的数据增强，具体涉及基于生成对抗网络的图像数据智能增强技术，尤其是面向舰船目标检测的SAR图像数据增强方法。
背景技术
在海洋监测、地质勘探等领域中，合成孔径雷达(SAR)具有全天时、全天候、高分辨、宽测绘带等特点。强穿透的主动式微波遥感技术，是海洋信息获取与监测中不可或缺的重要手段合。由于SAR图像反映的是目标的电磁散射特性，其中包含的匀质杂波和人造杂波降低了传统目标检测方法的准确率。除了常见的杂波虚警之外，由舰船目标自身所导致的方位向模糊也是舰船目标检测中的一个难点问题。船只的方位向模糊主要是由船只的单次或者二次散射导致的。传统的舰船目标检测所利用的数据是单通道SAR数据，这类数据只包含舰船的强度信息，因此不能全面地反映船只目标与海面杂波间的散射信息差异。目前在SAR图像船舶检测领域应用最为广泛的是CFAR及其各种改进算法。它通过对背景杂波进行统计分布建模来检测舰船目标。由于SAR系统的相干成像特点以及海面状况的复杂性，导致SAR图像中杂波分布具有较强的时变性、非平稳性、非高斯性等特征，使得CFAR检测算法具有较高的虚警率。并且，CFAR检测算法在很大程度上依赖于对杂波模型、虚警率、目标窗口、保护窗口和背景窗口的准确估计，使得检测性能在实际应用中存在较大的不确定性。识别的数据集需要收集各类舰船目标的大量的样本，难度较大。数据集SSDD是通过在网上下载公开的SAR图像，并将目标区域裁剪成大小为500×500左右像素，并通过人工标注舰船目标位置而得的。SSDD是国内外公开的第一个专门用于SAR图像舰船目标检测的数据集，它可以用于训练和测试检测算法，使研究人员在同一个条件下对比算法性能。对于每个舰船，检测算法预测舰船目标的边框，并给出是舰船目标的置信度。SSDD是借鉴PASCAL VOC数据集的制作过程来制作的，这是因为PASCAL VOC在目标检测领域应用较多，数据格式较规范，可以直接使用现有的算法在SSDD数据集上处理，对代码改动较小。传统的以CFAR为主的 检测算法能适应这类场景。复杂背景(靠岸区域)下的舰船目标尺寸小，这些目标背景复杂，传统方法要进行海陆分割才能进行检测，相比于基于深度学习的方法，会存在漏警和虚警的问题。靠近码头密集排列的大尺寸的舰船目标，此时传统检测方法难以检测到这些目标，而深度学习方法可以检测到它们。舰船目标长或者宽度所占图像尺寸的比例在0.04到0.24范围内，比PASCAL VOC的0.2到0.9要小很多。这为改进现有的深度学习目标检测算法提供了参考。通常，垂直边框中的很多像素不属于船的像素，这对于区分背景和舰船区域十分不利，尤其是密集排列的交叠非常大的舰船目标。舰船大多数是在45度左右的倾斜方向，其它几个方向基本上呈现的是均匀的分布，即各种旋转角度都会存在。通过对数据集中目标尺寸的统计分析可以看到相比于计算机视觉领域的数据集，SSDD中的目标尺寸很小，小尺寸检测一直是比较困难的(MS COCO中大尺寸的准确率比小尺寸高两倍左右)，因为它包含的信息少，不易提取的特征。随着深度学习技术的发展，其在SAR舰船检测领域的成功应用证明了深度神经网络模型挖掘目标特征的能力。然而，不可避免的问题是，深度神经网络模型需要大量数据支撑训练，而SAR数据的标注过于昂贵且耗时。
早在2014年，Ian GoodFellowes等人首次提出了生成对抗网络(Generative Adversarial Networks，GAN)用于生成新的数据样本。通过编码网络尽可能生成和真实样本相同的数据，对抗达到纳什均衡。但现有生成对抗网络方法中的目标函数难以有效指导梯度下降而造成训练不稳定甚至模式崩溃，为了解决梯度消失的问题，Xudong Mao等人在2017年提出了最小二乘生成对抗网络，采用最小二乘损失函数作为判别器，优化目标函数达到更好地生成高质量图像，且表现更为稳定。通常在处理两个及以上具有有限的可伸缩性和鲁棒性的图像域时，需要为每对图像域建立不同的模型。
随着硬件资源的更新换代，Andrew Brock以及DeepMind团队提出了BigGAN，其主要创新点在于将正交正则化的思想引入GAN，通过对输入先验分布z的适时截断使得训练更平稳；增加2～4倍的参数量和8倍的块大小，大大提升了GAN的生成性能。但是该模型计算量相当庞大，因此需要配合较高的硬件配置才能满足模型需求。而且传统GAN生成的SAR图像无标签，需人工打标后才能作为训练数据使用。
在SAR图像目标检测领域，现有数据集通用性低，且受轨道、空域等因素限制导致SAR图像数据在观察角度、分辨率等方面是离散且缺失的。现实环境中的多种因素影响了SAR图像目标检测的准确率。针对SAR图像质量不高、样本数量不充足等问题，部分学者引进GAN技术以生成新的SAR数据，增加样本多样性。张明蕊等人采用梯度惩罚WGAN (WGAN-GP)快速稳定的生成不同方位角的SAR图像。针对SAR图像极不匀质区域分割中的样本不平衡问题，有学者提出了结构约束的生成对抗网络，有效解决了GAN生成的图像结构模糊、严重变形的问题。然而，现有的一系列GAN增强SAR图像技术多为生成不带标签的数据，然后使用半监督模型进行分类或检测。虽然半监督模型的引入能简化生成模型的设计，但效果和有监督网络仍存在一定差距。如何构建自动生成带标签的SAR图像网络模型是极具实际应用价值的研究方向。常规的图像增强方法大多基于像素点的灰度值进行处理，这些方法对噪声较为敏感，而SAR图像中具有较强的相干斑噪声，往往不能有效地对“有用”区域进行增强。
随着海洋开发利用强度的与日俱增,以及海洋权属争端形势的日益严峻,海上舰船检测与动态监测应用对相关监视监控系统的“实时性”、“机动性”提出了更高的要求。卫星遥感技术是海上舰船监视监测的重要手段,在舰船检测、分类、识别与运动特征提取中发挥了重要的作用,然而低轨道卫星在观测范围、实时性、连续性动态观测方面已难以满足相关应用的需求。
发明内容
本公开的目的是针对现有SAR图像数据增强方法存在的生成数据质量低、无标签、实用性不强等问题，提供一种训练稳定、模式稳健，具有多样性及实用性，能够提升生成SAR图像数据的质量，面向舰船目标检测的SAR图像数据增强方法。
本公开的上述目的可以通过以下措施来实现，一种增强舰船目标检测SAR图像数据的方法，包括如下步骤：
首先，以舰船位置为中心，将图像形式的舰船位置作为SAR图像增强的约束条件c，约束条件c和隐变量z作为基于舰船位置信息的条件生成对抗网络生成器的输入，隐变量z通过两个全连接层后得到高维特征向量，并将其重构为隐变量特征图，约束条件c通过卷积层得到条件特征图，隐变量特征图和条件特征图级联后输入到至少4层转置卷积层，得到综合特征图，逐层上采样提高特征分辨率，生成新的SAR舰船图像，将作为约束条件c的目标框对应转换成所生成SAR数据的标签；其次，对于输入的真实SAR图像和生成的SAR图像，判别器结合作为约束条件的舰船位置信息，形成作为判别器网络输入的数据-标签对，判别器通过卷积层提取数据-标签对的特征，判断生成SAR图像的真假，同时判断生成图像/真实图像分别与对应约束条件的匹配程度，通过生成器和判别器的对抗学习，生成SAR图像更接近真实的SAR图像；最后，将所构建的条件生成对抗网络模型与目标检测网络进行级联，通过 舰船目标检测结果评估生成SAR图像数据质量，并将检测结果反馈给条件生成对抗网络，不断优化生成器，激励其生成更高质量的SAR图像新数据，通过对抗网络与目标检测网络的协同学习，增强SAR图像数据。
本公开相比于现有技术的有效增益在于：
(1)本公开面向舰船目标检测，以舰船位置为中心，将图像形式的舰船位置作为SAR图像增强的约束条件c，约束条件c和隐变量z作为基于舰船位置信息的条件生成对抗网络生成器的输入，隐变量z通过两个全连接层后得到高维特征向量，并将其重构为隐变量特征图，约束条件c通过卷积层得到条件特征图，隐变量特征图和条件特征图级联后输入到至少4层转置卷积层，得到综合特征图，逐层上采样提高特征分辨率，生成新的SAR舰船图像，将作为约束条件c的目标框对应转换成所生成SAR数据的标签；对于输入的真实SAR图像和生成的SAR图像，判别器结合作为约束条件的舰船位置信息，形成作为判别器网络输入的数据-标签对，判别器通过卷积层提取数据-标签对的特征，判断生成SAR图像的真假，同时判断生成图像/真实图像分别与对应约束条件的匹配程度。这种基于舰船位置信息辅助的条件生成对抗网络通过使用最优传输距离作为度量函数，可更好地衡量真实数据与生成数据间的距离，正确指导模型梯度下降方向，从而使模型的训练稳定、模式稳健。可以避免现有生成对抗网络方法中的目标函数难以有效指导梯度下降而造成训练不稳定甚至模式崩溃。
(2)本公开将隐变量特征图和条件特征图级联后输入到至少4层转置卷积层，得到综合特征图，逐层上采样提高特征分辨率，生成新的SAR舰船图像，将作为约束条件c的目标框对应转换成所生成SAR数据的标签，具备生成带标签SAR图像数据的能力。本公开的基于舰船位置信息辅助的条件生成对抗网络将舰船位置信息作为生成器的约束条件和生成图像的标签，在网络训练完成后，可直接生成新的带标签的SAR图像数据，克服了传统GAN生成的SAR图像无标签，需人工打标后才能作为训练数据使用的缺陷。
(3)本公开将所构建的基于舰船位置信息辅助的条件生成对抗网络模型与目标检测网络进行级联，通过舰船目标检测结果评估生成SAR图像数据质量，并将检测结果反馈给条件生成对抗网络，不断优化生成器，激励其生成更高质量的SAR图像新数据，通过对抗网络与目标检测网络的协同学习，增强SAR图像数据。具有多样性及实用性，能够提升生成SAR图像数据的质量。
附图说明
图1是本发明基于舰船位置信息的条件生成对抗网络结构示意图；
图2是本发明构建的条件生成对抗网络与目标检测协同学习框架示意图。
为使本发明的目的、技术方案和优点更加清楚明白，以下结合具体实施例，并参照附图，对本发明进一步详细说明。
具体实施方式
参阅图1。首先，以舰船位置为中心，将图像形式的舰船位置作为SAR图像增强的约束条件c，约束条件c和隐变量z作为基于舰船位置信息的条件生成对抗网络生成器的输入，隐变量z通过两个全连接层后得到高维特征向量，并将其重构为隐变量特征图，约束条件c通过卷积层得到条件特征图，隐变量特征图和条件特征图级联后输入到至少4层转置卷积层，得到综合特征图，逐层上采样提高特征分辨率，生成新的SAR舰船图像，将作为约束条件c的目标框对应转换成所生成SAR数据的标签；其次，对于输入的真实SAR图像和生成的SAR图像，判别器结合作为约束条件的舰船位置信息，形成作为判别器网络输入的数据-标签对，判别器通过卷积层提取数据-标签对的特征，判断生成SAR图像的真假，同时判断生成图像/真实图像分别与对应约束条件的匹配程度，通过生成器和判别器的对抗学习，生成SAR图像更接近真实的SAR图像；最后，将所构建的条件生成对抗网络模型与目标检测网络进行级联，通过舰船目标检测结果评估生成SAR图像数据质量，并将检测结果反馈给条件生成对抗网络，不断优化生成器，激励其生成更高质量的SAR图像新数据，通过对抗网络与目标检测网络的协同学习，增强SAR图像数据。
基于舰船位置信息的条件生成对抗网络，包括：生成器和判别器两部分，生成器以隐变量z和约束条件c作为输入，隐变量z通过两个全连接层后得到高维特征向量，并将其重构为4×4×256的隐变量特征图；约束条件c通过4层步长为2的卷积层得到4×6×256的条件特征图；将两个特征图级联后输入到4层转置卷积层，逐层上采样提高特征分辨率，生成大小为64×64像素的新的SAR舰船图像，并且为了有效利用特征信息，在每两层转置卷积层间增加一个残差连接。判别器通过图像-标签对的方式引入舰船位置信息，将生成图像/真实图像分别与对应的约束条件级联，形成64×64×26的网络输入；判别器网络结构由4个堆叠卷积层、两层全连接层和softmax层组成，卷积层提取数据-标签对的特征，全连接层将特征转化为标量，softmax层为线性分类器，可以鉴别生成SAR图像的真假，同时判断生成图像/真实图像分别与对应约束条件的匹配程度，进而实现对生成器生成结果的反馈调节，提高模型的生成质量。基于舰船位置信息的条件生成对抗网络引入最优传输距离构建生成器目标函数：
Figure PCTCN2022083836-appb-000001
和判别器的目标函数：
Figure PCTCN2022083836-appb-000002
其中，E[·]表示分布函数期望值，x～P r表示x服从概率分布P r，
Figure PCTCN2022083836-appb-000003
表示
Figure PCTCN2022083836-appb-000004
服从概率分布P g，c、
Figure PCTCN2022083836-appb-000005
分别为真实图像和生成图像对应约束，
Figure PCTCN2022083836-appb-000006
是真实图像的非匹配约束，D(x|c)和
Figure PCTCN2022083836-appb-000007
分别表示在约束c和约束
Figure PCTCN2022083836-appb-000008
下x是真实图像的概率，
Figure PCTCN2022083836-appb-000009
表示在约束
Figure PCTCN2022083836-appb-000010
下
Figure PCTCN2022083836-appb-000011
是生成图像的概率。
Figure PCTCN2022083836-appb-000012
表示判别器关于其输入的梯度，||·||表示矩阵的绝对值之和，r为真实数据与合成数据以一定比例混合(多为1:1)后的数据子集，x是从r中选取的数据样本，k和p是超参数。
基于舰船位置信息的条件生成对抗网络的总损失函数为L all＝L G+L D，通过梯度下降法分别求解生成器和判别器总损失函数L all的最优解G*和D*，G *,
Figure PCTCN2022083836-appb-000013
得到最优解，其中，
Figure PCTCN2022083836-appb-000014
表示对L all中的G和D求最大最小值。
参阅图2。基于舰船位置信息的条件生成对抗网络与目标检测网络连接，构建反馈自优化的SAR图像增强模型，主要由约束条件构建、基于舰船位置信息的条件生成对抗网络、增强SAR图像数据质量评估三部分组成。以真实SAR图像作为输入，通过约束条件构建模块生成舰船目标的位置约束，输入基于舰船位置信息的条件生成对抗网络生成SAR图像，通过YOLO V3目标检测网络模型评估生成SAR图像的质量，将舰船检测结果反馈给条件生成对抗网络，激励生成器生成质量更高的SAR图像，最终对抗生成网络与目标检测网络达到协同学习，形成一个有序的自主迭代循环优化机制。
以上所述的仅是本发明的优选实施例子。应当指出，对于本领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干变形和改进，如调整定位指的形状以适应特殊外形的连接器；或者修改定位件将定位装置安装在设备上，使其作为连接器阵列的固定工具；或者采用柔韧性材料制造定位指，使本装置兼具夹紧作用等，这些变更和改变均应视为属于本发明的保护范围。

Claims (9)

  1. 一种增强舰船目标检测SAR图像数据的方法，包括如下步骤：
    首先，以舰船位置为中心，将图像形式的舰船位置作为SAR图像增强的约束条件c，约束条件c和隐变量z作为基于舰船位置信息的条件生成对抗网络生成器的输入，隐变量z通过两个全连接层后得到高维特征向量，并将其重构为隐变量特征图，约束条件c通过卷积层得到条件特征图，隐变量特征图和条件特征图级联后输入到至少4层转置卷积层，得到综合特征图，逐层上采样提高特征分辨率，生成新的SAR舰船图像，将作为约束条件c的目标框对应转换成所生成SAR数据的标签；其次，对于输入的真实SAR图像和生成的SAR图像，判别器结合作为约束条件的舰船位置信息，形成作为判别器网络输入的数据-标签对，判别器通过卷积层提取数据-标签对的特征，判断生成SAR图像的真假，同时判断生成图像/真实图像分别与对应约束条件的匹配程度，通过生成器和判别器的对抗学习，生成SAR图像更接近真实的SAR图像；最后，将所构建的条件生成对抗网络模型与目标检测网络进行级联，通过舰船目标检测结果评估生成SAR图像数据质量，并将检测结果反馈给条件生成对抗网络，不断优化生成器，激励其生成更高质量的SAR图像新数据，通过对抗网络与目标检测网络的协同学习，增强SAR图像数据。
  2. 如权利要求1所述的增强舰船目标检测SAR图像数据的方法，其中，基于舰船位置信息的条件生成对抗网络，包括：生成器和判别器两部分，生成器以隐变量z和约束条件c作为输入，隐变量z通过两个全连接层后得到高维特征向量，并将其重构为4×4×256的隐变量特征图。
  3. 如权利要求1所述的增强舰船目标检测SAR图像数据的方法，其中，约束条件c通过4层步长为2的卷积层得到4×6×256的条件特征图；将两个特征图级联后输入到4层转置卷积层，逐层上采样提高特征分辨率，生成大小为64×64像素的新的SAR舰船图像。
  4. 如权利要求1所述的增强舰船目标检测SAR图像数据的方法，其中，为了有效利用特征信息，在每两层转置卷积层间增加一个残差连接。
  5. 如权利要求1所述的增强舰船目标检测SAR图像数据的方法，其中，判别器通过图像-标签对的方式引入舰船位置信息，将生成图像/真实图像分别与对应的约束条件级联，形成64×64×26的网络输入。
  6. 如权利要求1所述的增强舰船目标检测SAR图像数据的方法，其中，判别器网络结构由4个堆叠卷积层、两层全连接层和softmax层组成，卷积层提取数据-标签对的特征，全连接层将特征转化为标量，softmax层为线性分类器，可以鉴别生成SAR图像的真假，同时判断生 成图像/真实图像分别与对应约束条件的匹配程度，进而实现对生成器生成结果的反馈调节，提高模型的生成质量。
  7. 如权利要求1所述的增强舰船目标检测SAR图像数据的方法，其中，基于舰船位置信息的条件生成对抗网络引入最优传输距离构建生成器目标函数：
    Figure PCTCN2022083836-appb-100001
    和判别器的目标函数：
    Figure PCTCN2022083836-appb-100002
    其中，E[·]表示分布函数期望值，x～P r表示x服从概率分布P r，
    Figure PCTCN2022083836-appb-100003
    表示
    Figure PCTCN2022083836-appb-100004
    服从概率分布P g，c、
    Figure PCTCN2022083836-appb-100005
    分别为真实图像和生成图像对应约束，
    Figure PCTCN2022083836-appb-100006
    是真实图像的非匹配约束，D(x|c)和
    Figure PCTCN2022083836-appb-100007
    分别表示在约束c和约束
    Figure PCTCN2022083836-appb-100008
    下x是真实图像的概率，
    Figure PCTCN2022083836-appb-100009
    表示在约束
    Figure PCTCN2022083836-appb-100010
    下
    Figure PCTCN2022083836-appb-100011
    是生成图像的概率。
    Figure PCTCN2022083836-appb-100012
    表示判别器关于其输入的梯度，||·||表示矩阵的绝对值之和，r为真实数据与合成数据以一定比例混合(多为1:1)后的数据子集，
    Figure PCTCN2022083836-appb-100013
    是从r中选取的数据样本，k和p是超参数。
  8. 如权利要求1所述的增强舰船目标检测SAR图像数据的方法，其中，基于舰船位置信息的条件生成对抗网络的总损失函数为L all＝L G+L D，通过梯度下降法分别求解生成器和判别器总损失函数L all的最优解G*和D*，G *,
    Figure PCTCN2022083836-appb-100014
    得到最优解，其中，
    Figure PCTCN2022083836-appb-100015
    表示对L all中的G和D求最大最小值。
  9. 如权利要求1所述的增强舰船目标检测SAR图像数据的方法，其中，以真实SAR图像作为输入，通过约束条件构建模块生成舰船目标的位置约束，输入基于舰船位置信息的条件生成对抗网络生成SAR图像，通过YOLO V3目标检测网络模型评估生成SAR图像的质量，将舰船检测结果反馈给条件生成对抗网络，激励生成器生成质量更高的SAR图像，最终对抗生成网络与目标检测网络达到协同学习，形成一个有序的自主迭代循环优化机制。
PCT/CN2022/083836 2021-09-30 2022-03-29 增强舰船目标检测sar图像数据的方法 WO2023050746A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111159144.8A CN113962883B (zh) 2021-09-30 2021-09-30 增强舰船目标检测sar图像数据的方法
CN202111159144.8 2021-09-30

Publications (1)

Publication Number Publication Date
WO2023050746A1 true WO2023050746A1 (zh) 2023-04-06

Family

ID=79462768

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/083836 WO2023050746A1 (zh) 2021-09-30 2022-03-29 增强舰船目标检测sar图像数据的方法

Country Status (2)

Country Link
CN (1) CN113962883B (zh)
WO (1) WO2023050746A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116310837A (zh) * 2023-04-11 2023-06-23 安徽大学 一种sar舰船目标旋转检测方法及系统
CN116416136A (zh) * 2023-04-17 2023-07-11 北京卫星信息工程研究所 可见光遥感图像舰船目标检测的数据扩增方法、电子设备
CN116665095A (zh) * 2023-05-18 2023-08-29 中国科学院空间应用工程与技术中心 一种运动舰船检测方法、系统、存储介质和电子设备
CN117576553A (zh) * 2024-01-15 2024-02-20 中国海洋大学 双极化sar图像海洋冰涡识别方法、装置和电子设备

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113962883B (zh) * 2021-09-30 2023-02-03 西南电子技术研究所（中国电子科技集团公司第十研究所） 增强舰船目标检测sar图像数据的方法
CN114519679B (zh) * 2022-02-21 2022-10-21 安徽大学 一种sar目标图像数据智能增强方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109543740A (zh) * 2018-11-14 2019-03-29 哈尔滨工程大学 一种基于生成对抗网络的目标检测方法
US20200258296A1 (en) * 2019-02-08 2020-08-13 Ursa Space Systems Inc. Satellite sar artifact suppression for enhanced three-dimensional feature extraction, change detection, and visualizations
CN111781599A (zh) * 2020-07-16 2020-10-16 哈尔滨工业大学 基于CV-EstNet的SAR运动舰船目标速度估算方法
CN111899172A (zh) * 2020-07-16 2020-11-06 武汉大学 一种面向遥感应用场景的车辆目标检测方法
CN113962883A (zh) * 2021-09-30 2022-01-21 西南电子技术研究所（中国电子科技集团公司第十研究所） 增强舰船目标检测sar图像数据的方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112052899A (zh) * 2020-09-03 2020-12-08 上海无线电设备研究所 一种基于生成对抗网络的单幅舰船目标sar图像生成方法
CN112597815A (zh) * 2020-12-07 2021-04-02 西北工业大学 一种基于Group-G0模型的合成孔径雷达图像舰船检测方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109543740A (zh) * 2018-11-14 2019-03-29 哈尔滨工程大学 一种基于生成对抗网络的目标检测方法
US20200258296A1 (en) * 2019-02-08 2020-08-13 Ursa Space Systems Inc. Satellite sar artifact suppression for enhanced three-dimensional feature extraction, change detection, and visualizations
CN111781599A (zh) * 2020-07-16 2020-10-16 哈尔滨工业大学 基于CV-EstNet的SAR运动舰船目标速度估算方法
CN111899172A (zh) * 2020-07-16 2020-11-06 武汉大学 一种面向遥感应用场景的车辆目标检测方法
CN113962883A (zh) * 2021-09-30 2022-01-21 西南电子技术研究所（中国电子科技集团公司第十研究所） 增强舰船目标检测sar图像数据的方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
BARMPOUTIS PANAGIOTIS, PAPAIOANNOU PERIKLIS, DIMITROPOULOS KOSMAS, GRAMMALIDIS NIKOS: "A Review on Early Forest Fire Detection Systems Using Optical Remote Sensing", SENSORS, MDPI, CH, vol. 20, no. 22, 11 November 2020 (2020-11-11), CH , pages 1 - 26, XP055946762, ISSN: 1424-8220, DOI: 10.3390/s20226442 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116310837A (zh) * 2023-04-11 2023-06-23 安徽大学 一种sar舰船目标旋转检测方法及系统
CN116310837B (zh) * 2023-04-11 2024-04-23 安徽大学 一种sar舰船目标旋转检测方法及系统
CN116416136A (zh) * 2023-04-17 2023-07-11 北京卫星信息工程研究所 可见光遥感图像舰船目标检测的数据扩增方法、电子设备
CN116416136B (zh) * 2023-04-17 2023-12-19 北京卫星信息工程研究所 可见光遥感图像舰船目标检测的数据扩增方法、电子设备
CN116665095A (zh) * 2023-05-18 2023-08-29 中国科学院空间应用工程与技术中心 一种运动舰船检测方法、系统、存储介质和电子设备
CN116665095B (zh) * 2023-05-18 2023-12-22 中国科学院空间应用工程与技术中心 一种运动舰船检测方法、系统、存储介质和电子设备
CN117576553A (zh) * 2024-01-15 2024-02-20 中国海洋大学 双极化sar图像海洋冰涡识别方法、装置和电子设备
CN117576553B (zh) * 2024-01-15 2024-04-02 中国海洋大学 双极化sar图像海洋冰涡识别方法、装置和电子设备

Also Published As

Publication number Publication date
CN113962883B (zh) 2023-02-03
CN113962883A (zh) 2022-01-21

Similar Documents

Publication Publication Date Title
WO2023050746A1 (zh) 增强舰船目标检测sar图像数据的方法
CN109636742B (zh) 基于对抗生成网络的sar图像和可见光图像的模式转换方法
CN109871902B (zh) 一种基于超分辨率对抗生成级联网络的sar小样本识别方法
CN103955701B (zh) 多层次结合的多视合成孔径雷达图像目标识别方法
Qi et al. FTC-Net: Fusion of transformer and CNN features for infrared small target detection
CN115147731A (zh) 一种基于全空间编码注意力模块的sar图像目标检测方法
Xue et al. Dual network structure with interweaved global-local feature hierarchy for transformer-based object detection in remote sensing image
Zhou et al. PVT-SAR: An arbitrarily oriented SAR ship detector with pyramid vision transformer
Yasir et al. ShipGeoNet: SAR image-based geometric feature extraction of ships using convolutional neural networks
Li et al. Spatiotemporal remote-sensing image fusion with patch-group compressed sensing
Zhou et al. ISAR images generation via generative adversarial networks
Zhang et al. Swin-PAFF: A SAR Ship Detection Network with Contextual Cross-Information Fusion.
Zhao et al. Orientation-aware feature fusion network for ship detection in SAR images
Yan et al. Scheme to implement moving target detection of coastal defense radar in complicated sea conditions
Xu et al. SAR target recognition based on variational autoencoder
Han et al. Remote Sensing Teacher: Cross-Domain Detection Transformer with Learnable Frequency-Enhanced Feature Alignment in Remote Sensing Imagery
Li et al. PCViT: A Pyramid Convolutional Vision Transformer Detector for Object Detection in Remote Sensing Imagery
Yu et al. Dual-Resolution and Deformable Multihead Network for Oriented Object Detection in Remote Sensing Images
Zhong et al. Transfer learning based on joint feature matching and adversarial networks
Mu et al. Hyperspectral and panchromatic image fusion based on CNMF
Liang et al. SCDFMixer: Spatial-Channel Dual-Frequency Mixer Based on Satellite Optical Sensors for Remote Sensing Multi-Object Detection
Qi et al. Strip segmentation of oceanic internal waves in SAR images based on TransUNet
CN108985445A (zh) 一种基于机器学习理论的sar目标方位鉴别方法
Baiqi et al. Aircraft Rotated Boxes Detection Method Based on YOLOv5
Li et al. Prompt Learning for Oriented Power Transmission Tower Detection in High-Resolution SAR Images

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22874147

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WA Withdrawal of international application
122 Ep: pct application non-entry in european phase

Ref document number: 22874147

Country of ref document: EP

Kind code of ref document: A1