CN111860178A - A small sample remote sensing target detection method and system based on weight dictionary learning - Google Patents

A small sample remote sensing target detection method and system based on weight dictionary learning Download PDF

Info

Publication number
CN111860178A
CN111860178A CN202010576615.4A CN202010576615A CN111860178A CN 111860178 A CN111860178 A CN 111860178A CN 202010576615 A CN202010576615 A CN 202010576615A CN 111860178 A CN111860178 A CN 111860178A
Authority
CN
China
Prior art keywords
dictionary
target
target detection
remote sensing
detection model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010576615.4A
Other languages
Chinese (zh)
Other versions
CN111860178B (en
Inventor
陈凯强
张跃
许光銮
张腾飞
戴威
王雅珊
周琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aerospace Information Research Institute of CAS
Original Assignee
Aerospace Information Research Institute of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aerospace Information Research Institute of CAS filed Critical Aerospace Information Research Institute of CAS
Priority to CN202010576615.4A priority Critical patent/CN111860178B/en
Publication of CN111860178A publication Critical patent/CN111860178A/en
Application granted granted Critical
Publication of CN111860178B publication Critical patent/CN111860178B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/28—Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本发明公开了一种基于权重字典学习的小样本遥感目标检测方法及系统，所述方法获取待分类的遥感图像数据，将数据带入预先训练的目标检测模型中得到所述遥感图像对应的目标类别，所述目标检测模型利用小样本数据基于权重字典学习训练得到。所述方法采用权重字典学习的方式构建了轻量化的小样本遥感目标检测模型，可有效降低可学习参数数量，防止模型在小数据下训练时产生过拟合，提高了模型的小样本学习性能；又能很好的保留模型在源域上学习到的知识，避免了灾难性遗忘的问题。本发明提出的基于权重字典的遥感目标检测方法具有很好的通用性，可被用于改进其他的基于深度学习的遥感目标检测模型，提高它们的小样本学习能力。

Figure 202010576615

The invention discloses a small sample remote sensing target detection method and system based on weight dictionary learning. The method obtains remote sensing image data to be classified, and brings the data into a pre-trained target detection model to obtain the target corresponding to the remote sensing image. category, the target detection model is obtained by learning and training based on a weight dictionary using small sample data. The method constructs a lightweight small-sample remote sensing target detection model by means of weight dictionary learning, which can effectively reduce the number of learnable parameters, prevent the model from overfitting when training under small data, and improve the small-sample learning performance of the model. ; and can well retain the knowledge learned by the model on the source domain, avoiding the problem of catastrophic forgetting. The weight dictionary-based remote sensing target detection method proposed in the present invention has good versatility, and can be used to improve other remote sensing target detection models based on deep learning and improve their small sample learning ability.

Figure 202010576615

Description

一种基于权重字典学习的小样本遥感目标检测方法及系统A small sample remote sensing target detection method and system based on weight dictionary learning

技术领域technical field

本发明涉及遥感图像目标检测，具体涉及一种基于权重字典学习的小样本遥感目标检测方法及系统。The invention relates to remote sensing image target detection, in particular to a small sample remote sensing target detection method and system based on weight dictionary learning.

背景技术Background technique

自动化遥感图像目标检测技术可以自动地定位、识别静态遥感图像中的感兴趣目标。基于深度学习的遥感图像目标检测方法取得了飞速地发展，但是，这类基于深度学习的遥感图像目标检测方法仍存在一定的局限性。Automatic remote sensing image target detection technology can automatically locate and identify interesting targets in static remote sensing images. Remote sensing image target detection methods based on deep learning have achieved rapid development, but such deep learning-based remote sensing image target detection methods still have certain limitations.

基于深度学习的遥感图像目标检测模型依赖大量的训练样本。这些模型只有在大量的训练样本上经过数万次甚至更多的训练迭代才能取得很好的性能，而当训练样本不足时，这些模型很容易发生过拟合，在测试数据上的性能会变差。况且，收集大量的训练样本，并对这些样本进行标注，非常地费时费力，而且一些目标，如新型号的飞机，可能没有足够的样本来构建数据集，这就使得基于深度学习的遥感图像目标检测方法难以应用于这些样本不足的目标。此外，现实中的视觉概念往往服从长尾分布，即人们普遍关注的视觉概念样本是比较重组的，凡是新兴的视觉概念在不断地增加，这些新兴视觉概念的样本往往很少，因此基于深度学习的目标检测方法也很难应用于这些新兴视觉概念。Remote sensing image target detection models based on deep learning rely on a large number of training samples. These models can achieve good performance only after tens of thousands or even more training iterations on a large number of training samples. When the training samples are insufficient, these models are prone to overfitting, and the performance on the test data will change. Difference. Moreover, collecting a large number of training samples and labeling these samples is very time-consuming and labor-intensive, and some targets, such as new models of aircraft, may not have enough samples to construct a dataset, which makes the deep learning-based remote sensing image targets. Detection methods are difficult to apply to these undersampled targets. In addition, the visual concepts in reality often obey the long-tail distribution, that is, the visual concept samples that people generally pay attention to are relatively reorganized, and the emerging visual concepts are constantly increasing, and the samples of these emerging visual concepts are often very few. Therefore, based on deep learning It is also difficult to apply the object detection methods of 1 to these emerging vision concepts.

基于深度学习的遥感图像目标检测模型的任务扩展性很差。这些模型是在含有一组固定目标类别的训练集上训练的，将模型部署到应用环境中以后，模型无法检测在训练集中没有出现过的新的目标类别。为了让模型能对新的目标类别进行有效地检测，需要采集这些新类别的样本，然后进行样本标注，将这些训练数据加入原数据集中，对模型进行重新训练或微调模型的部分参数。但是上述过程是非常的费时费力的，而且新的目标类别样本数量不一定充足，这就使得基于深度学习的遥感目标检测模型很难有效扩展到对新类别目标的检测任务中。The task scalability of deep learning-based remote sensing image object detection models is poor. These models are trained on a training set containing a fixed set of target classes. After the model is deployed in the application environment, the model cannot detect new target classes that did not appear in the training set. In order for the model to effectively detect new target categories, it is necessary to collect samples of these new categories, then label the samples, add these training data to the original data set, and retrain the model or fine-tune some parameters of the model. However, the above process is very time-consuming and labor-intensive, and the number of samples of new target categories is not necessarily sufficient, which makes it difficult to effectively extend the remote sensing target detection model based on deep learning to the detection task of new categories of targets.

发明内容SUMMARY OF THE INVENTION

为了解决基于深度学习的遥感目标检测模型依赖大量训练数据以及对新任务扩展性差的问题，本发明提供一种基于权重字典学习的小样本遥感目标检测方法，包括：In order to solve the problem that the deep learning-based remote sensing target detection model relies on a large amount of training data and has poor scalability to new tasks, the present invention provides a small sample remote sensing target detection method based on weight dictionary learning, including:

获取待分类的遥感图像数据；Obtain remote sensing image data to be classified;

将所述数据带入预先训练的目标检测模型中得到所述遥感图像对应的目标类别；Bringing the data into a pre-trained target detection model to obtain a target category corresponding to the remote sensing image;

其中，所述目标检测模型利用小样本数据基于权重字典学习训练得到。The target detection model is obtained by learning and training based on a weight dictionary using small sample data.

优选的，所述目标检测模型的训练包括：Preferably, the training of the target detection model includes:

基于带有目标类别的历史遥感图像数据构建目标检测数据集；Build a target detection dataset based on historical remote sensing image data with target categories;

将所述遥感图像目标检测数据集划分为源类数据集与目标类数据集；dividing the remote sensing image target detection data set into a source data set and a target data set;

利用所述源数据集进行训练得到单阶段目标检测模型，并基于所述单阶段目标模型的卷积层参数构建参数字典，为参数字典中的每个参数设置一个对应的字典系数，基于所述参数字典与对应的字典系数构建基于权重字典的目标检测模型；A single-stage target detection model is obtained by training with the source data set, and a parameter dictionary is constructed based on the convolutional layer parameters of the single-stage target model, and a corresponding dictionary coefficient is set for each parameter in the parameter dictionary. The parameter dictionary and the corresponding dictionary coefficients construct the target detection model based on the weight dictionary;

利用所述目标类数据集对所述基于权重字典的目标检测模型进行训练，得到最优的目标检测模型；Use the target class data set to train the weight dictionary-based target detection model to obtain the optimal target detection model;

优选的，所述将所述遥感图像目标检测数据集划分为源类数据集与目标类数据集，包括：Preferably, dividing the remote sensing image target detection data set into a source data set and a target data set, including:

将遥感图像目标检测数据集中的目标类别划分为源类与目标类；Divide the target classes in the remote sensing image target detection dataset into source classes and target classes;

将同时包含源类和目标类的目标的遥感图像从数据集中丢弃；Discard remote sensing images of targets that contain both source and target classes from the dataset;

对数据集中剩余的遥感图像，将仅包含源类目标的图像划分为源数据集，将仅包含目标类目标的图像划分为目标数据集；For the remaining remote sensing images in the dataset, the images containing only the source class targets are divided into the source data set, and the images only containing the target class targets are divided into the target data set;

优选的，所述利用所述源数据集进行训练得到单阶段目标检测模型，并基于所述单阶段目标模型的卷积层参数构建参数字典，为参数字典中的每个参数设置一个对应的字典系数，基于所述参数字典与对应的字典系数构建基于字典的目标检测模型，包括：Preferably, the single-stage target detection model is obtained by using the source data set for training, and a parameter dictionary is constructed based on the convolutional layer parameters of the single-stage target model, and a corresponding dictionary is set for each parameter in the parameter dictionary coefficients, and a dictionary-based target detection model is constructed based on the parameter dictionary and the corresponding dictionary coefficients, including:

将所述源数据集划分为训练集和测试集；dividing the source data set into a training set and a test set;

利用所述训练集中的样本对单阶段目标检测模型Ds进行训练，并利用测试集中的样本进行测试，直至在测试集所有的样本上达到最好的测试性能；Use the samples in the training set to train the single-stage target detection model D s , and use the samples in the test set for testing, until the best test performance is achieved on all the samples in the test set;

然后以所述单阶段目标检测Ds除最后用于确定目标类别和位置的层外的所有卷积层参数φ作为参数字典；Then use all the convolutional layer parameters φ of the single-stage target detection Ds except the last layer used to determine the target category and position as the parameter dictionary;

对参数字典φ中的每个字典参数设置一个对应的字典系数w；其中字典系数的初值随机确定；Set a corresponding dictionary coefficient w for each dictionary parameter in the parameter dictionary φ; the initial value of the dictionary coefficient is randomly determined;

使用由所有卷积层参数φ构成的参数字典与对应的字典系数w构建一个基于字典的目标检测模型Dd；Construct a dictionary-based target detection model D d using the parameter dictionary composed of all convolutional layer parameters φ and the corresponding dictionary coefficients w;

其中，所述参数字典φ是固定的，字典系数w和目标检测模型Dd确定的分类、回归层参数θ可以被修改，且字典系数w的参数量远小于参数字典的参数量。The parameter dictionary φ is fixed, the classification and regression layer parameters θ determined by the dictionary coefficient w and the target detection model D d can be modified, and the parameter quantity of the dictionary coefficient w is much smaller than that of the parameter dictionary.

优选的，所述参数字典φ中的参数由所有的卷积层的参数构成；Preferably, the parameters in the parameter dictionary φ are composed of parameters of all convolutional layers;

参数字典φ中每个卷积层的参数是形状为C×N×k×k的张量。The parameters of each convolutional layer in the parameter dictionary φ are tensors of shape C×N×k×k.

优选的，所述使用由所有卷积层参数φ构成的参数字典与对应的字典系数w构建一个基于字典的目标检测模型Dd，包括：Preferably, a dictionary-based target detection model D d is constructed by using the parameter dictionary composed of all convolutional layer parameters φ and the corresponding dictionary coefficients w, including:

以参数字典φ中一个形状为C×N×k×k的初始卷积层Convs为字典；Take an initial convolutional layer Conv s of shape C×N×k×k in the parameter dictionary φ as a dictionary;

将初始卷积层Convs分解为C个形状为N×k×k的子张量；Decompose the initial convolutional layer Conv s into C sub-tensors of shape N×k×k;

对初始卷积层Convs中所有子张量进行线性组合构成目标卷积层中的每个子张量Td，以每个子张量为卷积核，并为每个卷积核建立字典系数；Linearly combine all sub-tensors in the initial convolutional layer Conv s to form each sub-tensor T d in the target convolutional layer, take each sub-tensor as a convolution kernel, and establish a dictionary coefficient for each convolution kernel;

基于子张量Td和对应的字典系数构建目标卷积层Convd，其中，目标卷积层Convd的形状与初始卷积层Convs的形状相同；Construct the target convolutional layer Conv d based on the sub-tensor T d and the corresponding dictionary coefficients, wherein the shape of the target convolutional layer Conv d is the same as the shape of the initial convolutional layer Conv s ;

由目标卷积层Convd构建目标检测模型Dd。The object detection model D d is constructed by the object convolutional layer Conv d .

优选的，所述目标卷积层Convd中各字典系数的构建过程如下公式：Preferably, the construction process of each dictionary coefficient in the target convolution layer Conv d is as follows:

Figure BDA0002551280770000041
Figure BDA0002551280770000041

式中，

Figure BDA0002551280770000042
表示新卷积层Convd中第i个卷积核对应于参数字典中卷积层Convs中第j个卷积核的字典系数。In the formula,
Figure BDA0002551280770000042
Indicates that the ith convolution kernel in the new convolutional layer Conv d corresponds to the dictionary coefficient of the jth convolution kernel in the convolutional layer Conv s in the parameter dictionary.

优选的，所述子张量Td的表达式如下：Preferably, the expression of the sub-tensor T d is as follows:

Figure BDA0002551280770000043
Figure BDA0002551280770000043

其中，wc表示对应第c个子张量的字典系数。Among them, w c represents the dictionary coefficient corresponding to the c-th sub-tensor.

优选的，所述利用所述目标类数据集对所述基于字典的目标检测模型进行训练，得到最优目标检测模型，包括：Preferably, the dictionary-based target detection model is trained by using the target class data set to obtain an optimal target detection model, including:

在目标数据集上划分目标训练集和目标测试集；Divide the target training set and the target test set on the target data set;

利用所述目标训练集的样本，对基于字典的遥感目标检测模型进行训练，优化其字典系数w和Dd中最后用于确定目标类别、目标位置的参数θ；Using the samples of the target training set, the dictionary-based remote sensing target detection model is trained, and the parameter θ that is finally used to determine the target category and target position in the dictionary coefficients w and D d is optimized;

利用目标测试集中的样本对优化后的基于字典的遥感目标检测模型进行测试确定小样本条件下的遥感目标检测模型Dd；Using the samples in the target test set to test the optimized dictionary-based remote sensing target detection model to determine the remote sensing target detection model D d under the condition of small samples;

优选的，所述字典系数优化的目标函数如下：Preferably, the objective function of the dictionary coefficient optimization is as follows:

Figure BDA0002551280770000044
Figure BDA0002551280770000044

其中，w表示字典系数，θ表示模型Dd用于回归、分类的卷积层

Figure BDA0002551280770000045
和
Figure BDA0002551280770000046
的参数，I表示输入图像，
Figure BDA0002551280770000047
与
Figure BDA0002551280770000048
分别表别标签和位置标签。Among them, w represents the dictionary coefficient, and θ represents the convolutional layer of the model D d used for regression and classification
Figure BDA0002551280770000045
and
Figure BDA0002551280770000046
The parameters of , I represent the input image,
Figure BDA0002551280770000047
and
Figure BDA0002551280770000048
Identify labels and location labels, respectively.

优选的，所述利用所述目标类数据集对所述基于字典的目标检测模型进行训练，得到最优目标检测模型，还包括：Preferably, the dictionary-based target detection model is trained by using the target class data set to obtain an optimal target detection model, further comprising:

对所述目标数据集进行多次的划分目标训练集和目标测试集；The target data set is divided into a target training set and a target test set multiple times;

针对每次划分的目标训练集和目标测试集对所述基于字典的目标检测模型进行训练；The dictionary-based target detection model is trained for each divided target training set and target test set;

对多次训练中的测试结果进行评估，以评估的平均值作为最终的测试评估结果。The test results in multiple trainings are evaluated, and the average value of the evaluation is used as the final test evaluation result.

基于同一种发明构思，本发明还提供一种基于权重字典学习的小样本遥感目标检测系统，包括：Based on the same inventive concept, the present invention also provides a small sample remote sensing target detection system based on weight dictionary learning, including:

数据获取模块，用于获取待分类的遥感图像数据；A data acquisition module for acquiring remote sensing image data to be classified;

目标检测模块，用于将所述数据带入预先训练的目标检测模型中得到所述遥感图像中遥感目标的位置和类别；A target detection module for bringing the data into a pre-trained target detection model to obtain the position and category of the remote sensing target in the remote sensing image;

其中，所述目标检测模型利用小样本数据基于权重字典学习训练得到。The target detection model is obtained by learning and training based on a weight dictionary using small sample data.

优选的：目标检测模型构建模块，用于利用小样本数据基于字典进行学习训练得到目标检测模型；Preferably: a target detection model building module, used for learning and training based on a dictionary using small sample data to obtain a target detection model;

优选的，所述目标检测模型构建模块包括：Preferably, the target detection model building module includes:

目标检测数据集构建单元，用于基于带有目标类别的历史遥感图像数据构建目标检测数据集；A target detection data set construction unit is used to construct a target detection data set based on historical remote sensing image data with target categories;

目标检测数据集划分单元，用于将所述遥感图像目标检测数据集划分为源类数据集与目标类数据集；a target detection data set dividing unit, configured to divide the remote sensing image target detection data set into a source data set and a target data set;

目标检测模型建立单元，用于利用所述源数据集进行训练得到单阶段目标检测模型，并基于所述单阶段目标模型的卷积层参数构建参数字典，为参数字典中的每个参数设置一个对应的字典系数，基于所述参数字典与对应的字典系数构建基于字典的目标检测模型；还用于：利用所述目标类数据集对所述基于权重字典的目标检测模型进行训练，得到最优的目标检测模型。A target detection model establishment unit is used to obtain a single-stage target detection model by using the source data set for training, and build a parameter dictionary based on the convolutional layer parameters of the single-stage target model, and set a parameter for each parameter in the parameter dictionary. The corresponding dictionary coefficients are used to construct a dictionary-based target detection model based on the parameter dictionary and the corresponding dictionary coefficients; it is also used for: using the target class data set to train the weight dictionary-based target detection model to obtain an optimal object detection model.

与现有技术相比，本发明的有益效果为：Compared with the prior art, the beneficial effects of the present invention are:

1、本发明提供一种基于权重字典学习的小样本遥感目标检测方法，包括：获取待分类的遥感图像数据；将所述数据带入预先训练的目标检测模型中得到所述遥感图像对应的目标类别；其中，所述目标检测模型利用小样本数据基于权重字典学习训练得到，与现有的基于迁移学习的小样本遥感目标检测方法相比，本发明提出的方法既能有效降低可学习参数数量，又能很好的保留模型在源域上学习到的知识，避免了灾难性遗忘的问题。1. The present invention provides a small sample remote sensing target detection method based on weight dictionary learning, comprising: obtaining remote sensing image data to be classified; bringing the data into a pre-trained target detection model to obtain a target corresponding to the remote sensing image. The target detection model is obtained by using small sample data to learn and train based on a weight dictionary. Compared with the existing small sample remote sensing target detection methods based on migration learning, the method proposed by the present invention can effectively reduce the number of parameters that can be learned. , and can well retain the knowledge learned by the model in the source domain, avoiding the problem of catastrophic forgetting.

2、本发明提出的方法采用权重字典学习的方式构建了轻量化的小样本遥感目标检测模型，可有效防止模型在小数据下训练时产生过拟合，提高了模型的小样本学习性能。2. The method proposed in the present invention constructs a lightweight small-sample remote sensing target detection model by means of weight dictionary learning, which can effectively prevent the model from overfitting during training with small data, and improve the small-sample learning performance of the model.

3、本发明提出的基于权重字典的遥感目标检测方法具有很好的通用性，可被用于改进其他的基于深度学习的遥感目标检测模型，提高它们的小样本学习能力。3. The weight dictionary-based remote sensing target detection method proposed by the present invention has good versatility, and can be used to improve other remote sensing target detection models based on deep learning and improve their small sample learning ability.

附图说明Description of drawings

图1为本发明基于权重字典学习的小样本遥感目标检测方法流程图；1 is a flowchart of a small sample remote sensing target detection method based on weight dictionary learning of the present invention;

图2为本申请实施例提供的一种基于权重字典学习的小样本遥感目标检测方法中训练过程示意图；2 is a schematic diagram of a training process in a small sample remote sensing target detection method based on weight dictionary learning provided by an embodiment of the present application;

图3为本申请实施例提供的基于权重字典学习的小样本遥感目标检测的数据集划分流程示意图；3 is a schematic flowchart of a data set division process for small-sample remote sensing target detection based on weight dictionary learning provided by an embodiment of the present application;

图4为本申请实施例提供的一种基于权重字典学习的小样本遥感目标检测框架示意图；4 is a schematic diagram of a small sample remote sensing target detection framework based on weight dictionary learning provided by an embodiment of the present application;

图5为本申请实施例提供的字典学习原理示意图；5 is a schematic diagram of a dictionary learning principle provided by an embodiment of the present application;

图6为本发明提供的一种基于权重字典学习的小样本遥感目标检测系统示意图。FIG. 6 is a schematic diagram of a small sample remote sensing target detection system based on weight dictionary learning provided by the present invention.

具体实施方式Detailed ways

为了更好地理解本发明，下面结合说明书附图和实例对本发明的内容做进一步的说明。In order to better understand the present invention, the content of the present invention will be further described below with reference to the accompanying drawings and examples.

实施例1：Example 1:

本发明提供一种基于权重字典学习的小样本遥感目标检测方法，如图1所示，包括：The present invention provides a small sample remote sensing target detection method based on weight dictionary learning, as shown in Figure 1, including:

获取待分类的遥感图像数据；Obtain remote sensing image data to be classified;

将所述数据带入预先训练的目标检测模型中得到所述遥感图像对应的目标类别；Bringing the data into a pre-trained target detection model to obtain a target category corresponding to the remote sensing image;

其中，所述目标检测模型利用小样本数据基于权重字典学习训练得到。The target detection model is obtained by learning and training based on a weight dictionary using small sample data.

这里目标检测模型的训练如图2所示，包括：The training of the target detection model here is shown in Figure 2, including:

(1)基于带有目标类别的历史遥感图像数据构建目标检测数据集；(1) Construct a target detection dataset based on historical remote sensing image data with target categories;

(2)将所述遥感图像目标检测数据集划分为源类数据集与目标类数据集；(2) dividing the remote sensing image target detection data set into a source data set and a target data set;

(3)利用所述源数据集进行训练得到单阶段目标检测模型，并基于所述单阶段目标模型的卷积层参数构建参数字典，为参数字典中的每个参数设置一个权重作为对应的字典系数，基于所述参数字典与对应的字典系数构建基于权重字典的目标检测模型；(3) Using the source data set for training to obtain a single-stage target detection model, and constructing a parameter dictionary based on the convolutional layer parameters of the single-stage target model, setting a weight for each parameter in the parameter dictionary as the corresponding dictionary coefficients, constructing a weight dictionary-based target detection model based on the parameter dictionary and the corresponding dictionary coefficients;

(4)利用所述目标类数据集对所述基于字典的目标检测模型进行训练，得到最优的目标检测模型。(4) Using the target class data set to train the dictionary-based target detection model to obtain an optimal target detection model.

其中，(2)将所述遥感图像目标检测数据集划分为源类数据集与目标类数据集，如图3所示，具体包括：Wherein, (2) the remote sensing image target detection data set is divided into a source class data set and a target class data set, as shown in Figure 3, specifically including:

步骤S1：将遥感图像目标检测数据集中的目标类别划分为源类与目标类；Step S1: Divide the target categories in the remote sensing image target detection data set into source categories and target categories;

步骤S2：按照遥感图像中包含的目标类别对其进行筛选：对于只包含源类目标的遥感图像，将其划分到源数据集；对于只包含目标类目标的遥感图像，将其划分到目标数据集；将同时包含源类和目标类的目标的遥感图像从数据集中丢弃，保证源类数据集与目标类数据集中的目标类别和数据均不相同；这里遥感地物目标类别包括但不限于：飞机、车辆、船舶、油罐、污水处理厂、篮球场、足球场、网球场、飞机场、火车站、桥梁、港口、立交桥、十字路口等。Step S2: Screen the remote sensing images according to the target categories contained in them: for remote sensing images containing only source targets, divide them into source datasets; for remote sensing images containing only target targets, divide them into target data Discard remote sensing images containing targets of both source and target classes from the dataset to ensure that the target categories and data in the source and target datasets are not the same; here, the target categories of remote sensing objects include but are not limited to: Aircraft, vehicles, ships, oil tanks, sewage treatment plants, basketball courts, football fields, tennis courts, airports, railway stations, bridges, ports, overpasses, intersections, etc.

步骤S3：对数据集中剩余的遥感图像，将仅包含源类目标的图像划分为源数据集，将仅包含目标类目标的图像划分为目标数据集；Step S3: for the remaining remote sensing images in the dataset, the images containing only the source class targets are divided into source data sets, and the images only containing the target class targets are divided into target data sets;

(3)利用所述源数据集进行训练得到单阶段目标检测模型，并基于所述单阶段目标模型的卷积层参数构建参数字典，为参数字典中的每个参数设置一个权重作为对应的字典系数，基于所述参数字典与对应的字典系数构建基于字典的目标检测模型。(3) Using the source data set for training to obtain a single-stage target detection model, and constructing a parameter dictionary based on the convolutional layer parameters of the single-stage target model, setting a weight for each parameter in the parameter dictionary as the corresponding dictionary coefficients, and a dictionary-based target detection model is constructed based on the parameter dictionary and the corresponding dictionary coefficients.

本实施例在源数据集的训练集上训练目标检测模型，该目标检测模型由特征提取器和单阶段目标检测器构成，其全部网络层均为卷积层。该训练过程与标准的深度学习目标目标检测模型相同，使用训练集中的全部样本训练模型，直到该模型在源数据集的测试集上达到最好的性能。In this embodiment, a target detection model is trained on the training set of the source data set. The target detection model is composed of a feature extractor and a single-stage target detector, and all network layers thereof are convolutional layers. The training process is the same as a standard deep learning target detection model, using all samples in the training set to train the model until the model achieves the best performance on the test set of the source data set.

如图4所示，具体包括：As shown in Figure 4, it specifically includes:

步骤S1：将源数据集划分为训练集和测试集，训练集和测试集共包含Csource个目标类别，源数据集的训练集中第i个目标类别

Figure BDA0002551280770000081
至少包含
Figure BDA0002551280770000082
(一般地，设
Figure BDA0002551280770000083
)个训练样本，即源数据集的整个训练集样本数量为
Figure BDA0002551280770000084
Step S1: Divide the source data set into a training set and a test set. The training set and the test set contain C source target categories in total, and the ith target category in the training set of the source data set
Figure BDA0002551280770000081
contains at least
Figure BDA0002551280770000082
(Generally, let
Figure BDA0002551280770000083
) training samples, that is, the number of samples in the entire training set of the source dataset is
Figure BDA0002551280770000084

步骤S2：源数据集的测试集中第i个目标类别

Figure BDA0002551280770000085
至少包含
Figure BDA0002551280770000086
(一般地，设
Figure BDA0002551280770000087
)个测试样本，即整个源数据集的测试集样本数量
Figure BDA0002551280770000088
Step S2: the i-th target category in the test set of the source dataset
Figure BDA0002551280770000085
contains at least
Figure BDA0002551280770000086
(Generally, let
Figure BDA0002551280770000087
) test samples, that is, the number of test set samples for the entire source dataset
Figure BDA0002551280770000088

步骤S3：对每个输入图像，目标检测模型首先检测到所有目标的位置，然后将每个目标分类到数据集中的一个类别上，如图像中包含飞机、船舶类的目标，则目标检测模型会检测到每个目标的位置，然后将其分类为飞机或船舶。在源数据集上，使用

Figure BDA0002551280770000089
个充足的训练样本训练单阶段目标检测模型，直至在
Figure BDA00025512807700000810
个测试集样本上达到最好的测试性能，然后以该模型Ds除最后用于确定目标类别和位置的层外的所有卷积层参数φ作为参数字典；Step S3: For each input image, the target detection model first detects the positions of all targets, and then classifies each target into a category in the dataset. For example, if the image contains targets of aircraft and ships, the target detection model will The location of each target is detected and then classified as an aircraft or a ship. On the source dataset, use
Figure BDA0002551280770000089
sufficient training samples to train a single-stage object detection model until
Figure BDA00025512807700000810
The best test performance is achieved on the test set samples, and then all the convolutional layer parameters φ of the model D s except the last layer used to determine the target category and position are used as the parameter dictionary;

(4)利用所述目标类数据集对所述基于字典的目标检测模型进行训练，得到最优的目标检测模型，如图5所示，具体包括：(4) Use the target class data set to train the dictionary-based target detection model to obtain the optimal target detection model, as shown in Figure 5, which specifically includes:

步骤S1：使用由φ构成的参数字典与对应的字典系数w构建一个基于字典的目标检测模型Dd，其中参数字典φ是固定的，仅有字典系数w和Dd中最后的分类、回归层参数θ可被修改，且字典系数w的参数量远小于参数字典的参数量(Num(w)＜＜Num(φ))。因此，相比原模型Ds的可学习参数{φ，θ}，模型Dd的可学习参数{w，θ}参数量要更少，即Num(Dd)＜＜Num(Ds)，即基于权重字典学习的模型Dd是一个轻量化的目标检测模型。于参数字典φ在源数据上的遥感目标检测任务上进行了训练，因此其包含丰富的遥感领域知识。此外，当源数据集中的遥感目标检测样本有限时，可在遥感图像地物分类数据上对参数字典进行训练，以确保参数字典具有遥感领域的知识。Step S1: Use the parameter dictionary composed of φ and the corresponding dictionary coefficient w to construct a dictionary-based target detection model D d , where the parameter dictionary φ is fixed, only the dictionary coefficient w and the last classification and regression layer in D d The parameter θ can be modified, and the parameter amount of the dictionary coefficient w is much smaller than that of the parameter dictionary (Num(w)<<Num(φ)). Therefore, compared with the learnable parameters {φ, θ} of the original model D s , the learnable parameters {w, θ} of the model D d are less, that is, Num(D d )<<Num(D s ), That is, the model D d based on weight dictionary learning is a lightweight target detection model. The parameter dictionary φ is trained on the remote sensing object detection task on the source data, so it contains rich remote sensing domain knowledge. In addition, when the remote sensing target detection samples in the source dataset are limited, the parameter dictionary can be trained on the remote sensing image feature classification data to ensure that the parameter dictionary has knowledge in the remote sensing domain.

参数字典φ中的参数：Parameters in parameter dictionary φ:

参数字典φ中的参数由其中所有的卷积层参数构成。The parameters in the parameter dictionary φ consist of all the convolutional layer parameters in it.

在该含有L个卷积层的参数字典中，第l(l∈[1,L])个卷积层为：

Figure BDA00025512807700000910
其中l表示层数，s表示该卷积层是在源数据集上训练的。参数字典φ中每个卷积层的参数是形状为C×N×k×k的张量，其中C表示该卷积层输出通道数量，N表示该卷积层输入通道数量，k表示卷积核的大小。以参数字典φ中一个形状为C×N×k×k的卷积层Convs为字典，可以构建一个新的卷积层，新卷积层Convd的形状与原卷积层Convs的形状相同。将Convs这个形状为C×N×k×k的张量分解为C个形状为N×k×k的子张量，第c个子张量记为Ts c。对新卷积层Convd，同样可分解为C个形状为N×k×k的子张量，第c个子张量记为Td c。新卷积层的每个子张量Td由原卷积层Convs中所有子张量的线性组合构成：
Figure BDA0002551280770000091
其中，wc表示对应第c个子张量的字典系数也即权重。使用所有的新卷积层，并在其后添加用于预测目标边界回归的卷积层
Figure BDA0002551280770000092
和预测目标类别的卷积层
Figure BDA0002551280770000093
构建一个基于字典的目标检测模型Dd。综上所述，新卷积层Convd的构建过程如下公式：In the parameter dictionary containing L convolutional layers, the l(l∈[1,L])th convolutional layer is:
Figure BDA00025512807700000910
where l represents the number of layers and s represents that the convolutional layer was trained on the source dataset. The parameters of each convolutional layer in the parameter dictionary φ are tensors of shape C×N×k×k, where C represents the number of output channels of the convolutional layer, N represents the number of input channels of the convolutional layer, and k represents the convolutional layer. the size of the nucleus. Taking a convolutional layer Conv s with a shape of C×N×k×k in the parameter dictionary φ as a dictionary, a new convolutional layer can be constructed. The shape of the new convolutional layer Conv d is the same as the shape of the original convolutional layer Conv s . same. Decompose Conv s , a tensor of shape C×N×k×k, into C sub-tensors of shape N×k×k, and the c-th sub-tensor is denoted as T s c . For the new convolutional layer Conv d , it can also be decomposed into C sub-tensors of shape N×k×k, and the c-th sub-tensor is denoted as T d c . Each sub-tensor T d of the new convolutional layer consists of a linear combination of all sub-tensors in the original convolutional layer Conv s :
Figure BDA0002551280770000091
Among them, w c represents the dictionary coefficient corresponding to the c-th sub-tensor, that is, the weight. Use all the new convolutional layers and add the convolutional layer after it for predicting the regression of the target boundary
Figure BDA0002551280770000092
and a convolutional layer that predicts the target class
Figure BDA0002551280770000093
Build a dictionary-based object detection model D d . To sum up, the construction process of the new convolutional layer Conv d is as follows:

Figure BDA0002551280770000094
Figure BDA0002551280770000094

Figure BDA0002551280770000095
表示新卷积层Convd中第i个卷积核对应于参数字典中卷积层Convs中第j个卷积核的字典系数。
Figure BDA0002551280770000095
Indicates that the ith convolution kernel in the new convolutional layer Conv d corresponds to the dictionary coefficient of the jth convolution kernel in the convolutional layer Conv s in the parameter dictionary.

步骤S2：在目标数据集上，共包含Ctarget个目标样本，且与源域数据集中的目标类别Csource不同，即

Figure BDA0002551280770000096
训练集中第i个目标类别
Figure BDA0002551280770000097
至多包含
Figure BDA0002551280770000098
(一般地，设
Figure BDA0002551280770000099
)个训练样本，即源数据集的整个训练集样本数量为
Figure BDA0002551280770000101
)；因此对该模型，只有字典参数和用于回归、分类的卷积层
Figure BDA0002551280770000102
和
Figure BDA0002551280770000103
的参数参与训练，从而减少了参与训练的参数数量。Step S2: On the target data set, a total of C target target samples are included, which are different from the target category C source in the source domain data set, that is,
Figure BDA0002551280770000096
The i-th target class in the training set
Figure BDA0002551280770000097
contains at most
Figure BDA0002551280770000098
(Generally, let
Figure BDA0002551280770000099
) training samples, that is, the number of samples in the entire training set of the source dataset is
Figure BDA0002551280770000101
); so for this model, there are only dictionary parameters and convolutional layers for regression and classification
Figure BDA0002551280770000102
and
Figure BDA0002551280770000103
The number of parameters involved in training, thereby reducing the number of parameters involved in training.

步骤S3：在目标数据集的测试集中第i个目标类别

Figure BDA0002551280770000104
至少包含
Figure BDA0002551280770000105
(一般地，设
Figure BDA0002551280770000106
)个测试样本，即目标据集的测试集样本数量为
Figure BDA0002551280770000107
)；Step S3: the i-th target category in the test set of the target dataset
Figure BDA0002551280770000104
contains at least
Figure BDA0002551280770000105
(Generally, let
Figure BDA0002551280770000106
) test samples, that is, the number of test set samples of the target data set is
Figure BDA0002551280770000107
);

步骤S4：在目标数据集的训练集中，仅使用

Figure BDA0002551280770000108
个少量的训练样本，训练基于字典的遥感目标检测模型，优化其字典系数w和Dd中最后的分类、回归层参数θ，实现小样本条件下的遥感目标检测模型Dd的训练和测试，字典系数优化的目标函数如下：Step S4: In the training set of the target dataset, only use
Figure BDA0002551280770000108
A small number of training samples are used to train a dictionary-based remote sensing target detection model, and the dictionary coefficient w and the final classification and regression layer parameters θ in D d are optimized to realize the training and testing of the remote sensing target detection model D d under the condition of small samples. The objective function of dictionary coefficient optimization is as follows:

Figure BDA0002551280770000109
Figure BDA0002551280770000109

其中，w表示字典系数，θ表示模型Dd用于回归、分类的卷积层

Figure BDA00025512807700001010
和
Figure BDA00025512807700001011
的参数，I表示输入图像，
Figure BDA00025512807700001012
与
Figure BDA00025512807700001013
分别表别标签和位置标签。由于参数字典包含丰富的遥感领域知识，因此基于参数字典构建的模型可有效实现对新类别遥感目标的小样本检测。Among them, w represents the dictionary coefficient, and θ represents the convolutional layer of the model D d used for regression and classification
Figure BDA00025512807700001010
and
Figure BDA00025512807700001011
The parameters of , I represent the input image,
Figure BDA00025512807700001012
and
Figure BDA00025512807700001013
Identify labels and location labels, respectively. Since the parameter dictionary contains rich remote sensing domain knowledge, the model constructed based on the parameter dictionary can effectively realize the small sample detection of new categories of remote sensing targets.

基于上述的目标函数，在目标数据集中训练集上的少量样本上训练基于字典的目标检测模型Dd，对其字典系数w和回归、分类层参数θ进行优化，然后在目标数据集的测试集上进行测试，这样便完成了小样本条件下的遥感图像目标检测模型Dd的训练与测试。Based on the above objective function, a dictionary-based target detection model D d is trained on a small number of samples in the training set of the target data set, and its dictionary coefficient w and regression and classification layer parameters θ are optimized, and then the test set of the target data set is used. In this way, the training and testing of the remote sensing image target detection model D d under the condition of small samples is completed.

此外，考虑到目标数据集的训练集中样本数量较少，不具有代表性，为了让测试结果更为可靠，一般将目标数据集重复划分M次，然后分别进行模型Dd的训练、测试，最后将M次划分中测试结果的平均值作为最终的测试结果。In addition, considering that the number of samples in the training set of the target data set is small and not representative, in order to make the test results more reliable, the target data set is generally divided M times repeatedly, and then the model D d is trained and tested respectively, and finally The average value of the test results in the M divisions is taken as the final test result.

实施例2，Example 2,

为了实现上述方法，本发明还提供一种基于权重字典学习的小样本遥感目标检测系统，如图6所示，包括：In order to realize the above method, the present invention also provides a small sample remote sensing target detection system based on weight dictionary learning, as shown in FIG. 6 , including:

数据获取模块，用于获取待分类的遥感图像数据；A data acquisition module for acquiring remote sensing image data to be classified;

目标检测模块，用于将所述数据带入预先由目标检测模型构建模块训练的目标检测模型中得到所述遥感图像中遥感目标的位置和类别；A target detection module for bringing the data into a target detection model trained in advance by the target detection model building module to obtain the position and category of the remote sensing target in the remote sensing image;

目标检测模型构建模块，用于利用小样本数据基于字典进行学习训练得到目标检测模型。The target detection model building module is used for learning and training based on a dictionary with small sample data to obtain a target detection model.

目标检测模型构建模块包括：Object detection model building blocks include:

目标检测数据集构建单元，用于基于带有目标类别的历史遥感图像数据构建目标检测数据集；A target detection data set construction unit is used to construct a target detection data set based on historical remote sensing image data with target categories;

目标检测数据集划分单元，用于将所述遥感图像目标检测数据集划分为源类数据集与目标类数据集；a target detection data set dividing unit, configured to divide the remote sensing image target detection data set into a source data set and a target data set;

目标检测模型建立单元，用于利用所述源数据集进行训练得到单阶段目标检测模型，并基于所述单阶段目标模型的卷积层参数构建参数字典，为参数字典中的每个参数设置一个对应的字典系数，基于所述参数字典与对应的字典系数构建基于字典的目标检测模型；还用于：利用所述目标类数据集对所述基于字典的目标检测模型进行训练，得到最优的目标检测模型。A target detection model establishment unit is used to obtain a single-stage target detection model by using the source data set for training, and build a parameter dictionary based on the convolutional layer parameters of the single-stage target model, and set a parameter for each parameter in the parameter dictionary. The corresponding dictionary coefficients are used to construct a dictionary-based target detection model based on the parameter dictionary and the corresponding dictionary coefficients; it is also used for: using the target class data set to train the dictionary-based target detection model to obtain an optimal object detection model.

其中，目标检测数据集划分单元，具体包括：Among them, the target detection data set division unit specifically includes:

将遥感图像目标检测数据集中的目标类别划分为源类与目标类；Divide the target classes in the remote sensing image target detection dataset into source classes and target classes;

将同时包含源类和目标类的目标的遥感图像从数据集中丢弃，保证源类数据集与目标类数据集中的目标类别和数据均不相同；这里遥感地物目标类别包括但不限于：飞机、车辆、船舶、油罐、污水处理厂、篮球场、足球场、网球场、飞机场、火车站、桥梁、港口、立交桥、十字路口等。Discard remote sensing images containing targets of both source and target classes from the dataset to ensure that the target classes and data in the source class dataset and the target class dataset are different; here, the target classes of remote sensing objects include but are not limited to: aircraft, Vehicles, ships, oil tanks, sewage treatment plants, basketball courts, football fields, tennis courts, airports, railway stations, bridges, ports, overpasses, intersections, etc.

对数据集中剩余的遥感图像，将仅包含源类目标的图像划分为源数据集，将仅包含目标类目标的图像划分为目标数据集；For the remaining remote sensing images in the dataset, the images containing only the source class targets are divided into the source data set, and the images only containing the target class targets are divided into the target data set;

目标检测模型建立单元，具体包括：Target detection model establishment unit, including:

将源数据集划分为训练集和测试集，训练集和测试集共包含Csource个目标类别，源数据集的训练集中第i个目标类别

Figure BDA0002551280770000111
至少包含
Figure BDA0002551280770000112
(一般地，设
Figure BDA0002551280770000113
)个训练样本，即源数据集的整个训练集样本数量为
Figure BDA0002551280770000114
Divide the source data set into training set and test set. The training set and test set contain C source target categories in total, and the ith target category in the training set of the source data set
Figure BDA0002551280770000111
contains at least
Figure BDA0002551280770000112
(Generally, let
Figure BDA0002551280770000113
) training samples, that is, the number of samples in the entire training set of the source dataset is
Figure BDA0002551280770000114

源数据集的测试集中第i个目标类别

Figure BDA0002551280770000115
至少包含
Figure BDA0002551280770000116
(一般地，设
Figure BDA0002551280770000117
)个测试样本，即整个源数据集的测试集样本数量
Figure BDA0002551280770000118
The i-th target category in the test set of the source dataset
Figure BDA0002551280770000115
contains at least
Figure BDA0002551280770000116
(Generally, let
Figure BDA0002551280770000117
) test samples, that is, the number of test set samples for the entire source dataset
Figure BDA0002551280770000118

对每个输入图像，目标检测模型首先检测到所有目标的位置，然后将每个目标分类到数据集中的一个类别上，如图像中包含飞机、船舶类的目标，则目标检测模型会检测到每个目标的位置，然后将其分类为飞机或船舶。在源数据集上，使用

Figure BDA0002551280770000121
个充足的训练样本训练单阶段目标检测模型，直至在
Figure BDA0002551280770000122
个测试集样本上达到最好的测试性能，然后以该模型Ds除最后用于确定目标类别和位置的层外的所有卷积层参数φ作为参数字典；For each input image, the target detection model first detects the positions of all targets, and then classifies each target into a category in the dataset. location of a target and then classify it as an aircraft or a ship. On the source dataset, use
Figure BDA0002551280770000121
sufficient training samples to train a single-stage object detection model until
Figure BDA0002551280770000122
The best test performance is achieved on the test set samples, and then all the convolutional layer parameters φ of the model D s except the last layer used to determine the target category and position are used as the parameter dictionary;

使用由φ构成的参数字典与对应的字典系数w构建一个基于字典的目标检测模型Dd，其中参数字典φ是固定的，仅有字典系数w和Dd中最后的分类、回归层参数θ可被修改，且字典系数w的参数量远小于参数字典的参数量(Num(w)＜＜Num(φ))。因此，相比原模型Ds的可学习参数{φ，θ}，模型Dd的可学习参数{w，θ}参数量要更少，即Num(Dd)＜＜Num(Ds)，即基于权重字典学习的模型Dd是一个轻量化的目标检测模型。于参数字典φ在源数据上的遥感目标检测任务上进行了训练，因此其包含丰富的遥感领域知识。此外，当源数据集中的遥感目标检测样本有限时，可在遥感图像地物分类数据上对参数字典进行训练，以确保参数字典具有遥感领域的知识。Use the parameter dictionary composed of φ and the corresponding dictionary coefficient w to construct a dictionary-based target detection model D d , where the parameter dictionary φ is fixed, only the dictionary coefficient w and the last classification and regression layer parameter θ in D d can be used. is modified, and the parameter quantity of the dictionary coefficient w is much smaller than that of the parameter dictionary (Num(w)<<Num(φ)). Therefore, compared with the learnable parameters {φ, θ} of the original model D s , the learnable parameters {w, θ} of the model D d are less, that is, Num(D d )<<Num(D s ), That is, the model D d based on weight dictionary learning is a lightweight target detection model. The parameter dictionary φ is trained on the remote sensing object detection task on the source data, so it contains rich remote sensing domain knowledge. In addition, when the remote sensing target detection samples in the source dataset are limited, the parameter dictionary can be trained on the remote sensing image feature classification data to ensure that the parameter dictionary has knowledge in the remote sensing field.

参数字典φ中的参数：Parameters in parameter dictionary φ:

参数字典φ中的参数由其中所有的卷积层参数构成。The parameters in the parameter dictionary φ consist of all the convolutional layer parameters in it.

参数字典φ中每个卷积层的参数是形状为C×N×k×k的张量，其中C表示该卷积层输出通道数量，N表示该卷积层输入通道数量，k表示卷积核的大小。以参数字典φ中一个形状为C×N×k×k的卷积层Convs为字典，可以构建一个新的卷积层，新卷积层Convd的形状与原卷积层Convs的形状相同。将Convs这个形状为C×N×k×k的张量分解为C个形状为N×k×k的子张量，第c个子张量记为Ts c。对新卷积层Convd，同样可分解为C个形状为N×k×k的子张量，第c个子张量记为Td c。新卷积层的每个子张量Td由原卷积层Convs中所有子张量的线性组合构成：

Figure BDA0002551280770000131
其中，wc表示对应第c个子张量的字典系数。综上所述，新卷积层Convd的构建过程如下公式：The parameters of each convolutional layer in the parameter dictionary φ are tensors of shape C×N×k×k, where C represents the number of output channels of the convolutional layer, N represents the number of input channels of the convolutional layer, and k represents the convolutional layer. the size of the nucleus. Taking a convolutional layer Conv s with a shape of C×N×k×k in the parameter dictionary φ as a dictionary, a new convolutional layer can be constructed. The shape of the new convolutional layer Conv d is the same as the shape of the original convolutional layer Conv s . same. Decompose Conv s , a tensor of shape C×N×k×k, into C sub-tensors of shape N×k×k, and the c-th sub-tensor is denoted as T s c . For the new convolutional layer Conv d , it can also be decomposed into C sub-tensors of shape N×k×k, and the c-th sub-tensor is denoted as T d c . Each sub-tensor T d of the new convolutional layer consists of a linear combination of all sub-tensors in the original convolutional layer Conv s :
Figure BDA0002551280770000131
Among them, w c represents the dictionary coefficient corresponding to the c-th sub-tensor. To sum up, the construction process of the new convolutional layer Conv d is as follows:

Figure BDA0002551280770000132
Figure BDA0002551280770000132

Figure BDA0002551280770000133
表示新卷积层Convd中第i个卷积核对应于参数字典中卷积层Convs中第j个卷积核的字典系数。
Figure BDA0002551280770000133
Indicates that the ith convolution kernel in the new convolutional layer Conv d corresponds to the dictionary coefficient of the jth convolution kernel in the convolutional layer Conv s in the parameter dictionary.

在目标数据集上，共包含Ctarget个目标样本，且与源域数据集中的目标类别Csource不同，即

Figure BDA0002551280770000134
训练集中第i个目标类别
Figure BDA0002551280770000135
至多包含
Figure BDA0002551280770000136
(一般地，设
Figure BDA0002551280770000137
)个训练样本，即源数据集的整个训练集样本数量为
Figure BDA0002551280770000138
On the target dataset, a total of C target samples are included, which are different from the target category C source in the source domain dataset, that is,
Figure BDA0002551280770000134
The i-th target class in the training set
Figure BDA0002551280770000135
contains at most
Figure BDA0002551280770000136
(Generally, let
Figure BDA0002551280770000137
) training samples, that is, the number of samples in the entire training set of the source dataset is
Figure BDA0002551280770000138

在目标数据集的测试集中第i个目标类别

Figure BDA0002551280770000139
至少包含
Figure BDA00025512807700001310
(一般地，设
Figure BDA00025512807700001311
)个测试样本，即目标据集的测试集样本数量为
Figure BDA00025512807700001312
i-th target class in the test set of the target dataset
Figure BDA0002551280770000139
contains at least
Figure BDA00025512807700001310
(Generally, let
Figure BDA00025512807700001311
) test samples, that is, the number of test set samples of the target data set is
Figure BDA00025512807700001312

步骤S4：在目标数据集的训练集中，仅使用

Figure BDA00025512807700001313
个少量的训练样本，训练基于字典的遥感目标检测模型，优化其字典系数w和Dd中最后的分类、回归层参数θ，实现小样本条件下的遥感目标检测模型Dd的训练和测试，字典系数优化的目标函数如下：Step S4: In the training set of the target dataset, only use
Figure BDA00025512807700001313
A small number of training samples are used to train a dictionary-based remote sensing target detection model, and the dictionary coefficient w and the final classification and regression layer parameters θ in D d are optimized to realize the training and testing of the remote sensing target detection model D d under the condition of small samples. The objective function of dictionary coefficient optimization is as follows:

Figure BDA00025512807700001314
Figure BDA00025512807700001314

其中，w表示字典系数，θ表示模型Dd用于回归、分类的卷积层

Figure BDA00025512807700001315
和
Figure BDA00025512807700001316
的参数，I表示输入图像，
Figure BDA00025512807700001317
与
Figure BDA00025512807700001318
分别表别标签和位置标签。由于参数字典包含丰富的遥感领域知识，因此基于参数字典构建的模型可有效实现对新类别遥感目标的小样本检测。Among them, w represents the dictionary coefficient, and θ represents the convolutional layer of the model D d used for regression and classification
Figure BDA00025512807700001315
and
Figure BDA00025512807700001316
The parameters of , I represent the input image,
Figure BDA00025512807700001317
and
Figure BDA00025512807700001318
Identify labels and location labels, respectively. Since the parameter dictionary contains rich remote sensing domain knowledge, the model constructed based on the parameter dictionary can effectively realize the small sample detection of new categories of remote sensing targets.

此外，考虑到目标数据集的训练集中样本数量较少，不具有代表性，为了让测试结果更为可靠，一般将目标数据集重复划分M次，然后分别进行模型Dd的训练、测试，最后将M次划分中测试结果的平均值作为最终的测试结果。In addition, considering that the number of samples in the training set of the target data set is small and not representative, in order to make the test results more reliable, the target data set is generally divided M times repeatedly, and then the model D d is trained and tested respectively, and finally The average value of the test results in the M divisions is taken as the final test result.

Claims (10)

1.一种基于权重字典学习的小样本遥感目标检测方法，其特征在于，包括：1. a small sample remote sensing target detection method based on weight dictionary learning, is characterized in that, comprises: 获取待分类的遥感图像数据；Obtain remote sensing image data to be classified; 将所述数据带入预先训练的目标检测模型中得到所述遥感图像对应的目标类别；Bringing the data into a pre-trained target detection model to obtain a target category corresponding to the remote sensing image; 其中，所述目标检测模型利用小样本数据基于权重字典学习训练得到。The target detection model is obtained by learning and training based on a weight dictionary using small sample data. 2.如权利要求1所述的目标检测方法，其特征在于，所述目标检测模型的训练包括：2. The target detection method according to claim 1, wherein the training of the target detection model comprises: 基于带有目标类别的历史遥感图像数据构建目标检测数据集；Build a target detection dataset based on historical remote sensing image data with target categories; 将所述遥感图像目标检测数据集划分为源类数据集与目标类数据集；dividing the remote sensing image target detection data set into a source data set and a target data set; 利用所述源数据集进行训练得到单阶段目标检测模型，并基于所述单阶段目标模型的卷积层参数构建参数字典，为参数字典中的每个参数设置一个对应的字典系数，基于所述参数字典与对应的字典系数构建基于权重字典的目标检测模型；A single-stage target detection model is obtained by training with the source data set, and a parameter dictionary is constructed based on the convolutional layer parameters of the single-stage target model, and a corresponding dictionary coefficient is set for each parameter in the parameter dictionary. The parameter dictionary and the corresponding dictionary coefficients construct the target detection model based on the weight dictionary; 利用所述目标类数据集对所述基于权重字典的目标检测模型进行训练，得到最优的目标检测模型；Use the target class data set to train the weight dictionary-based target detection model to obtain the optimal target detection model; 优选的，所述将所述遥感图像目标检测数据集划分为源类数据集与目标类数据集，包括：Preferably, dividing the remote sensing image target detection data set into a source data set and a target data set, including: 将遥感图像目标检测数据集中的目标类别划分为源类与目标类；Divide the target classes in the remote sensing image target detection dataset into source classes and target classes; 将同时包含源类和目标类的目标的遥感图像从数据集中丢弃；Discard remote sensing images of targets that contain both source and target classes from the dataset; 对数据集中剩余的遥感图像，将仅包含源类目标的图像划分为源数据集，将仅包含目标类目标的图像划分为目标数据集；For the remaining remote sensing images in the dataset, the images containing only the source class targets are divided into the source data set, and the images only containing the target class targets are divided into the target data set; 优选的，所述利用所述源数据集进行训练得到单阶段目标检测模型，并基于所述单阶段目标模型的卷积层参数构建参数字典，为参数字典中的每个参数设置一个对应的字典系数，基于所述参数字典与对应的字典系数构建基于权重字典的目标检测模型，包括：Preferably, the single-stage target detection model is obtained by using the source data set for training, and a parameter dictionary is constructed based on the convolutional layer parameters of the single-stage target model, and a corresponding dictionary is set for each parameter in the parameter dictionary coefficients, and build a weight dictionary-based target detection model based on the parameter dictionary and the corresponding dictionary coefficients, including: 将所述源数据集划分为训练集和测试集；dividing the source data set into a training set and a test set; 利用所述训练集中的样本对单阶段目标检测模型Ds进行训练，并利用测试集中的样本进行测试，直至在测试集所有的样本上达到最好的测试性能；Use the samples in the training set to train the single-stage target detection model D s , and use the samples in the test set for testing, until the best test performance is achieved on all the samples in the test set; 然后以所述单阶段目标检测Ds除最后用于确定目标类别和位置的层外的所有卷积层参数φ作为参数字典；Then use all the convolutional layer parameters φ of the single-stage target detection Ds except the last layer used to determine the target category and position as the parameter dictionary; 对参数字典φ中的每个字典参数设置一个对应的字典系数w；其中字典系数的初值随机确定；Set a corresponding dictionary coefficient w for each dictionary parameter in the parameter dictionary φ; the initial value of the dictionary coefficient is randomly determined; 使用由所有卷积层参数φ构成的参数字典与对应的字典系数w构建一个基于字典的目标检测模型Dd；Construct a dictionary-based target detection model D d using the parameter dictionary composed of all convolutional layer parameters φ and the corresponding dictionary coefficients w; 其中，所述参数字典φ是固定的，字典系数w和目标检测模型Dd确定的分类、回归层参数θ可以被修改，且字典系数w的参数量远小于参数字典的参数量。The parameter dictionary φ is fixed, the classification and regression layer parameters θ determined by the dictionary coefficient w and the target detection model D d can be modified, and the parameter quantity of the dictionary coefficient w is much smaller than that of the parameter dictionary. 3.如权利要求2所述的目标检测方法，其特征在于，所述参数字典φ中的参数由所有的卷积层的参数构成；3. The target detection method according to claim 2, wherein the parameters in the parameter dictionary φ are composed of parameters of all convolution layers; 参数字典φ中每个卷积层的参数是形状为C×N×k×k的张量。The parameters of each convolutional layer in the parameter dictionary φ are tensors of shape C×N×k×k. 4.如权利要求3所述的目标检测方法，其特征在于，所述使用由所有卷积层参数φ构成的参数字典与对应的字典系数w构建一个基于字典的目标检测模型Dd，包括：4. The target detection method as claimed in claim 3, wherein the use of the parameter dictionary formed by all convolutional layer parameters φ and the corresponding dictionary coefficients w builds a dictionary-based target detection model D d , comprising: 以参数字典φ中一个形状为C×N×k×k的初始卷积层Convs为字典；Take an initial convolutional layer Conv s of shape C×N×k×k in the parameter dictionary φ as a dictionary; 将初始卷积层Convs分解为C个形状为N×k×k的子张量；Decompose the initial convolutional layer Conv s into C sub-tensors of shape N×k×k; 对初始卷积层Convs中所有子张量进行线性组合构成目标卷积层中的每个子张量Td，以每个子张量为卷积核，并为每个卷积核建立字典系数；Linearly combine all sub-tensors in the initial convolutional layer Conv s to form each sub-tensor T d in the target convolutional layer, take each sub-tensor as a convolution kernel, and establish a dictionary coefficient for each convolution kernel; 基于子张量Td和对应的字典系数构建目标卷积层Convd，其中，目标卷积层Convd的形状与初始卷积层Convs的形状相同；Construct the target convolutional layer Conv d based on the sub-tensor T d and the corresponding dictionary coefficients, wherein the shape of the target convolutional layer Conv d is the same as the shape of the initial convolutional layer Conv s ; 由目标卷积层Convd构建目标检测模型Dd。The object detection model D d is constructed by the object convolutional layer Conv d . 5.如权利要求4所述的目标检测方法，其特征在于，所述目标卷积层Convd中各字典系数的构建过程如下公式：5. The target detection method according to claim 4, wherein the construction process of each dictionary coefficient in the target convolution layer Conv d is as follows:
Figure FDA0002551280760000031
Figure FDA0002551280760000031
 式中，
Figure FDA0002551280760000032
表示新卷积层Convd中第i个卷积核对应于参数字典中卷积层Convs中第j个卷积核的字典系数。In the formula,
Figure FDA0002551280760000032
Indicates that the ith convolution kernel in the new convolutional layer Conv d corresponds to the dictionary coefficient of the jth convolution kernel in the convolutional layer Conv s in the parameter dictionary. 6.如权利要求4或5所述的目标检测方法，其特征在于，所述子张量Td的表达式如下：6. The target detection method according to claim 4 or 5, wherein the expression of the sub-tensor T d is as follows:
Figure FDA0002551280760000033
Figure FDA0002551280760000033
 其中，wc表示对应第c个子张量的字典系数。Among them, w c represents the dictionary coefficient corresponding to the c-th sub-tensor. 7.如权利要求2所述的目标检测方法，其特征在于，所述利用所述目标类数据集对所述基于字典的目标检测模型进行训练，得到最优目标检测模型，包括：7. The target detection method of claim 2, wherein the dictionary-based target detection model is trained by using the target class data set to obtain an optimal target detection model, comprising: 在目标数据集上划分目标训练集和目标测试集；Divide the target training set and the target test set on the target data set; 利用所述目标训练集的样本，对基于字典的遥感目标检测模型进行训练，优化其字典系数w和Dd中最后用于确定目标类别、目标位置的参数θ；Using the samples of the target training set, the dictionary-based remote sensing target detection model is trained, and the parameter θ that is finally used to determine the target category and target position in the dictionary coefficients w and D d is optimized; 利用目标测试集中的样本对优化后的基于字典的遥感目标检测模型进行测试确定小样本条件下的遥感目标检测模型Dd；Using the samples in the target test set to test the optimized dictionary-based remote sensing target detection model to determine the remote sensing target detection model D d under the condition of small samples; 优选的，所述字典系数优化的目标函数如下：Preferably, the objective function of the dictionary coefficient optimization is as follows:
Figure FDA0002551280760000034
Figure FDA0002551280760000034
 其中，w表示字典系数，θ表示模型Dd用于回归、分类的卷积层
Figure FDA0002551280760000035
和
Figure FDA0002551280760000036
的参数，I表示输入图像，lcls与lreg分别表别标签和位置标签。Among them, w represents the dictionary coefficient, and θ represents the convolutional layer of the model D d used for regression and classification
Figure FDA0002551280760000035
and
Figure FDA0002551280760000036
The parameters of , I represent the input image, and l cls and l reg represent labels and position labels, respectively. 8.如权利要求7所述的目标检测方法，其特征在于，所述利用所述目标类数据集对所述基于字典的目标检测模型进行训练，得到最优目标检测模型，还包括：8. The target detection method according to claim 7, wherein the dictionary-based target detection model is trained by using the target class data set to obtain an optimal target detection model, further comprising: 对所述目标数据集进行多次的划分目标训练集和目标测试集；The target data set is divided into a target training set and a target test set multiple times; 针对每次划分的目标训练集和目标测试集对所述基于字典的目标检测模型进行训练；The dictionary-based target detection model is trained for each divided target training set and target test set; 对多次训练中的测试结果进行评估，以评估的平均值作为最终的测试评估结果。The test results in multiple trainings are evaluated, and the average value of the evaluation is used as the final test evaluation result. 9.一种基于权重字典学习的小样本遥感目标检测系统，其特征在于，包括：9. A small sample remote sensing target detection system based on weight dictionary learning, characterized in that, comprising: 数据获取模块，用于获取待分类的遥感图像数据；A data acquisition module for acquiring remote sensing image data to be classified; 目标检测模块，用于将所述数据带入预先训练的目标检测模型中得到所述遥感图像中遥感目标的位置和类别；A target detection module for bringing the data into a pre-trained target detection model to obtain the position and category of the remote sensing target in the remote sensing image; 其中，所述目标检测模型利用小样本数据基于权重字典学习训练得到。The target detection model is obtained by learning and training based on a weight dictionary using small sample data. 10.如权利要求9所述的目标检测系统，其特征在于，还包括：目标检测模型构建模块，用于利用小样本数据基于权重字典进行学习训练得到目标检测模型；10. The target detection system as claimed in claim 9, further comprising: a target detection model building module for using small sample data to perform learning and training based on a weight dictionary to obtain a target detection model; 优选的，所述目标检测模型构建模块包括：Preferably, the target detection model building module includes: 目标检测数据集构建单元，用于基于带有目标类别的历史遥感图像数据构建目标检测数据集；A target detection data set construction unit is used to construct a target detection data set based on historical remote sensing image data with target categories; 目标检测数据集划分单元，用于将所述遥感图像目标检测数据集划分为源类数据集与目标类数据集；a target detection data set dividing unit, configured to divide the remote sensing image target detection data set into a source data set and a target data set; 目标检测模型建立单元，用于利用所述源数据集进行训练得到单阶段目标检测模型，并基于所述单阶段目标模型的卷积层参数构建参数字典，为参数字典中的每个参数设置一个对应的字典系数，基于所述参数字典与对应的字典系数构建基于字典的目标检测模型；还用于：利用所述目标类数据集对所述基于权重字典的目标检测模型进行训练，得到最优的目标检测模型。A target detection model establishment unit is used to obtain a single-stage target detection model by using the source data set for training, and build a parameter dictionary based on the convolutional layer parameters of the single-stage target model, and set a parameter for each parameter in the parameter dictionary. The corresponding dictionary coefficients are used to construct a dictionary-based target detection model based on the parameter dictionary and the corresponding dictionary coefficients; it is also used for: using the target class data set to train the weight dictionary-based target detection model to obtain an optimal object detection model.
CN202010576615.4A 2020-06-22 2020-06-22 Small sample remote sensing target detection method and system based on weight dictionary learning Active CN111860178B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010576615.4A CN111860178B (en) 2020-06-22 2020-06-22 Small sample remote sensing target detection method and system based on weight dictionary learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010576615.4A CN111860178B (en) 2020-06-22 2020-06-22 Small sample remote sensing target detection method and system based on weight dictionary learning

Publications (2)

Publication Number Publication Date
CN111860178A true CN111860178A (en) 2020-10-30
CN111860178B CN111860178B (en) 2021-03-23

Family

ID=72988378

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010576615.4A Active CN111860178B (en) 2020-06-22 2020-06-22 Small sample remote sensing target detection method and system based on weight dictionary learning

Country Status (1)

Country Link
CN (1) CN111860178B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112686289A (en) * 2020-12-24 2021-04-20 微梦创科网络科技（中国）有限公司 Picture classification method and device
CN116912630A (en) * 2023-09-12 2023-10-20 深圳须弥云图空间科技有限公司 Target identification method and device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103258210A (en) * 2013-05-27 2013-08-21 中山大学 High-definition image classification method based on dictionary learning
CN103761531A (en) * 2014-01-20 2014-04-30 西安理工大学 Sparse-coding license plate character recognition method based on shape and contour features
CN104680182A (en) * 2015-03-09 2015-06-03 西安电子科技大学 Polarimetric SAR classification method on basis of NSCT and discriminative dictionary learning
CN110414616A (en) * 2019-08-02 2019-11-05 南京大学 A Remote Sensing Image Dictionary Learning Classification Method Using Spatial Relationship
CN111126287A (en) * 2019-12-25 2020-05-08 武汉大学 A deep learning detection method for dense targets in remote sensing images

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103258210A (en) * 2013-05-27 2013-08-21 中山大学 High-definition image classification method based on dictionary learning
CN103761531A (en) * 2014-01-20 2014-04-30 西安理工大学 Sparse-coding license plate character recognition method based on shape and contour features
CN104680182A (en) * 2015-03-09 2015-06-03 西安电子科技大学 Polarimetric SAR classification method on basis of NSCT and discriminative dictionary learning
CN110414616A (en) * 2019-08-02 2019-11-05 南京大学 A Remote Sensing Image Dictionary Learning Classification Method Using Spatial Relationship
CN111126287A (en) * 2019-12-25 2020-05-08 武汉大学 A deep learning detection method for dense targets in remote sensing images

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ZHANG Q: "Discriminative K-SVD for dictionary learning in face recognition", 《IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION（CVPR）》 *
李争名: "基于双权重约束的判别字典学习算法", 《计算机与数字工程》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112686289A (en) * 2020-12-24 2021-04-20 微梦创科网络科技（中国）有限公司 Picture classification method and device
CN116912630A (en) * 2023-09-12 2023-10-20 深圳须弥云图空间科技有限公司 Target identification method and device

Also Published As

Publication number Publication date
CN111860178B (en) 2021-03-23

Similar Documents

Publication Publication Date Title
CN111739075B (en) Deep network lung texture recognition method combining multi-scale attention
CN107704925B (en) Visual analysis system and method for deep neural network training process
CN109117883B (en) SAR image sea ice classification method and system based on long-time memory network
CN110135522B (en) Intelligent method for detecting and marking small target integration of remote sensing image
CN108875794B (en) An Image Visibility Detection Method Based on Transfer Learning
CN107016413B (en) A kind of online stage division of tobacco leaf based on deep learning algorithm
CN109472284A (en) A Cell Defect Classification Method Based on Unbiased Embedding Zero-Sample Learning
CN106776842A (en) Multi-medium data detection method and device
CN105574550A (en) Vehicle identification method and device
CN107004123A (en) Iterative defect filters out technique
CN108647702B (en) Large-scale food material image classification method based on transfer learning
CN108629369A (en) A kind of Visible Urine Sediment Components automatic identifying method based on Trimmed SSD
CN111985325B (en) Aerial small target rapid identification method in extra-high voltage environment evaluation
CN112489092A (en) Fine-grained industrial motion mode classification method, storage medium, equipment and device
CN111860106A (en) An unsupervised method for bridge crack identification
CN109102015A (en) A kind of SAR image change detection based on complex-valued neural networks
CN113095229B (en) Self-adaptive pedestrian re-identification system and method for unsupervised domain
Ridhovan et al. Disease detection in banana leaf plants using densenet and inception method
CN111860178A (en) A small sample remote sensing target detection method and system based on weight dictionary learning
CN117218434A (en) Concrete structure surface defect classification method and system based on hybrid neural network
CN114022446A (en) A leather defect detection method and system based on improved YOLOv3
CN116977710A (en) Remote sensing image long tail distribution target semi-supervised detection method
CN114677339A (en) Power transmission line bolt out-of-stock defect detection method introducing attention mechanism
CN111428655A (en) A scalp detection method based on deep learning
CN108256489B (en) Behavior prediction method and device based on deep reinforcement learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant