CN111860178B - Small sample remote sensing target detection method and system based on weight dictionary learning - Google Patents

Small sample remote sensing target detection method and system based on weight dictionary learning Download PDF

Info

Publication number
CN111860178B
CN111860178B CN202010576615.4A CN202010576615A CN111860178B CN 111860178 B CN111860178 B CN 111860178B CN 202010576615 A CN202010576615 A CN 202010576615A CN 111860178 B CN111860178 B CN 111860178B
Authority
CN
China
Prior art keywords
target
dictionary
target detection
detection model
parameter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010576615.4A
Other languages
Chinese (zh)
Other versions
CN111860178A (en
Inventor
陈凯强
张跃
许光銮
张腾飞
戴威
王雅珊
周琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aerospace Information Research Institute of CAS
Original Assignee
Aerospace Information Research Institute of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aerospace Information Research Institute of CAS filed Critical Aerospace Information Research Institute of CAS
Priority to CN202010576615.4A priority Critical patent/CN111860178B/en
Publication of CN111860178A publication Critical patent/CN111860178A/en
Application granted granted Critical
Publication of CN111860178B publication Critical patent/CN111860178B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/28—Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a small sample remote sensing target detection method and system based on weight dictionary learning. The method adopts a weight dictionary learning mode to construct a lightweight small sample remote sensing target detection model, can effectively reduce the number of learnable parameters, prevents the model from being over-fitted during training under small data, and improves the small sample learning performance of the model; and the knowledge learned by the model on the source domain can be well kept, and the problem of catastrophic forgetting is avoided. The remote sensing target detection method based on the weight dictionary has good universality, and can be used for improving other remote sensing target detection models based on deep learning and improving the small sample learning capability of the remote sensing target detection models.

Description

Small sample remote sensing target detection method and system based on weight dictionary learning
Technical Field
The invention relates to remote sensing image target detection, in particular to a small sample remote sensing target detection method and system based on weight dictionary learning.
Background
The automatic remote sensing image target detection technology can automatically position and identify interested targets in the static remote sensing image. The remote sensing image target detection method based on deep learning is developed rapidly, but the remote sensing image target detection method based on deep learning still has certain limitation.
The remote sensing image target detection model based on deep learning relies on a large number of training samples. These models can only achieve good performance after tens of thousands of training iterations or even more on a large number of training samples, and when the training samples are insufficient, the models are easy to over-fit, and the performance on the test data is deteriorated. Moreover, the collection of a large number of training samples and labeling of the samples are time-consuming and labor-consuming, and some targets, such as new airplanes, may not have enough samples to construct a data set, which makes the remote sensing image target detection method based on deep learning difficult to apply to targets with insufficient samples. In addition, real-world visual concepts are often subjected to long-tailed distribution, that is, samples of visual concepts generally concerned by people are relatively recombined, and as emerging visual concepts are continuously increased, samples of the emerging visual concepts are often few, so that the deep learning-based target detection method is difficult to apply to the emerging visual concepts.
The task expansibility of the remote sensing image target detection model based on deep learning is poor. These models are trained on a training set containing a fixed set of object classes, and after the models are deployed into an application environment, the models cannot detect new object classes that have not appeared in the training set. In order to enable the model to effectively detect the new target class, samples of the new class need to be collected, then sample labeling is carried out, the training data are added into the original data set, and the model is retrained or part of parameters of the model are fine-tuned. However, the above process is very time-consuming and labor-consuming, and the number of new target class samples is not necessarily sufficient, which makes it difficult to effectively extend the remote sensing target detection model based on deep learning to the task of detecting new class targets.
Disclosure of Invention
In order to solve the problems that a remote sensing target detection model based on deep learning depends on a large amount of training data and the expansibility of a new task is poor, the invention provides a small sample remote sensing target detection method based on weight dictionary learning, which comprises the following steps:
acquiring remote sensing image data to be classified;
bringing the data into a pre-trained target detection model to obtain a target class corresponding to the remote sensing image;
the target detection model is obtained by learning and training small sample data based on a weight dictionary.
Preferably, the training of the target detection model includes:
constructing a target detection data set based on historical remote sensing image data with target categories;
dividing the remote sensing image target detection data set into a source class data set and a target class data set;
training by using the source data set to obtain a single-stage target detection model, constructing a parameter dictionary based on convolutional layer parameters of the single-stage target model, setting a corresponding dictionary coefficient for each parameter in the parameter dictionary, and constructing a target detection model based on a weight dictionary based on the parameter dictionary and the corresponding dictionary coefficient;
training the target detection model based on the weight dictionary by using the target class data set to obtain an optimal target detection model;
preferably, the dividing the remote sensing image target detection data set into a source class data set and a target class data set includes:
dividing target classes in a remote sensing image target detection data set into a source class and a target class;
discarding the remote sensing image of the target simultaneously containing the source class and the target class from the data set;
for the residual remote sensing images in the data set, dividing the images only containing the source type targets into a source data set, and dividing the images only containing the target type targets into a target data set;
preferably, the training with the source data set to obtain a single-stage target detection model, constructing a parameter dictionary based on convolutional layer parameters of the single-stage target model, setting a corresponding dictionary coefficient for each parameter in the parameter dictionary, and constructing a dictionary-based target detection model based on the parameter dictionary and the corresponding dictionary coefficient includes:
dividing the source data set into a training set and a test set;
model D for single-stage target detection by using samples in training setsTraining, and testing by using the samples in the test set until the best test performance is achieved on all the samples in the test set;
then detecting D with the single-stage targetsAll the convolution layer parameters phi except the layer finally used for determining the object type and position are used as a parameter dictionary;
setting a corresponding dictionary coefficient w for each dictionary parameter in the parameter dictionary phi; wherein the initial value of the dictionary coefficient is randomly determined;
constructing a dictionary-based object detection model D using a parameter dictionary consisting of all convolutional layer parameters phi and corresponding dictionary coefficients wd；
Wherein the parameter dictionary phi is fixed, dictionary coefficients w and the target detection model DdThe determined classification, regression layer parameters theta may be modified and the parameter quantities of the dictionary coefficients w are much smaller than the parameter quantities of the parameter dictionary.
Preferably, the parameters in the parameter dictionary phi are composed of parameters of all convolution layers;
the parameters of each convolutional layer in the parameter dictionary phi are tensors of the shape C × N × k × k.
Preferably, the dictionary-based object detection model D is constructed by using a parameter dictionary composed of all convolution layer parameters phi and corresponding dictionary coefficients wdThe method comprises the following steps:
initial convolutional layer Conv with a shape of CxNxk x k in the parameter dictionary phisIs a dictionary;
conv of the initial convolution layersDecomposing into C sub-tensors of shape Nxk x k;
conv for initial convolutional layersAll ofThe sub-tensors are linearly combined to form each sub-tensor T in the target convolutional layerdTaking each sub tensor as a convolution kernel, and establishing a dictionary coefficient for each convolution kernel;
based on the sub-tensor TdConstruction of a target convolutional layer Conv with corresponding dictionary coefficientsdWherein the target convolutional layer ConvdShape of (2) and initial convolution layer ConvsAre the same in shape;
conv from the target convolutional layerdConstructing an object detection model Dd。
Preferably, the target convolutional layer ConvdThe construction process of each dictionary coefficient is as follows:
Figure BDA0002551280770000041
in the formula (I), the compound is shown in the specification,
Figure BDA0002551280770000042
conv indicating a new convolutional layerdThe ith convolution kernel corresponds to the convolution layer Conv in the parameter dictionarysDictionary coefficients of the jth convolution kernel.
Preferably, the sub-tensor TdThe expression of (a) is as follows:
Figure BDA0002551280770000043
wherein, wcDictionary coefficients representing the corresponding c-th sub-tensor.
Preferably, the training the dictionary-based target detection model by using the target class data set to obtain an optimal target detection model includes:
dividing a target training set and a target testing set on a target data set;
training a remote sensing target detection model based on a dictionary by using samples of the target training set, and optimizing dictionary coefficients w and D of the remote sensing target detection modeldFinally, determining a parameter theta of the target category and the target position;
testing the optimized remote sensing target detection model based on the dictionary by using the samples in the target test set to determine the remote sensing target detection model D under the condition of small samplesd；
Preferably, the objective function of the dictionary coefficient optimization is as follows:
Figure BDA0002551280770000044
where w represents dictionary coefficients and θ represents model DdConvolution layer for regression and classification
Figure BDA0002551280770000045
And
Figure BDA0002551280770000046
i denotes the input image,
Figure BDA0002551280770000047
and
Figure BDA0002551280770000048
respectively identify the tag and the location tag.
Preferably, the training the dictionary-based target detection model by using the target class data set to obtain an optimal target detection model further includes:
dividing the target data set into a target training set and a target testing set for multiple times;
training the dictionary-based target detection model aiming at the target training set and the target testing set which are divided each time;
and evaluating the test results in multiple training, and taking the average value of the evaluation as the final test evaluation result.
Based on the same inventive concept, the invention also provides a small sample remote sensing target detection system based on weight dictionary learning, which comprises:
the data acquisition module is used for acquiring remote sensing image data to be classified;
the target detection module is used for bringing the data into a pre-trained target detection model to obtain the position and the category of the remote sensing target in the remote sensing image;
the target detection model is obtained by learning and training small sample data based on a weight dictionary.
Preferably: the target detection model building module is used for performing learning training on the basis of a dictionary by using small sample data to obtain a target detection model;
preferably, the object detection model building module includes:
the target detection data set construction unit is used for constructing a target detection data set based on the historical remote sensing image data with the target category;
the target detection data set dividing unit is used for dividing the remote sensing image target detection data set into a source data set and a target data set;
the target detection model establishing unit is used for training by utilizing the source data set to obtain a single-stage target detection model, establishing a parameter dictionary based on the convolutional layer parameters of the single-stage target model, setting a corresponding dictionary coefficient for each parameter in the parameter dictionary, and establishing a dictionary-based target detection model based on the parameter dictionary and the corresponding dictionary coefficient; and is also used for: and training the target detection model based on the weight dictionary by using the target class data set to obtain an optimal target detection model.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention provides a small sample remote sensing target detection method based on weight dictionary learning, which comprises the following steps: acquiring remote sensing image data to be classified; bringing the data into a pre-trained target detection model to obtain a target class corresponding to the remote sensing image; compared with the existing small sample remote sensing target detection method based on transfer learning, the method provided by the invention can effectively reduce the quantity of learnable parameters, well reserve the knowledge learned by the model on the source domain and avoid the problem of catastrophic forgetting.
2. According to the method, the lightweight small sample remote sensing target detection model is constructed in a weight dictionary learning mode, overfitting of the model during training under small data can be effectively prevented, and the small sample learning performance of the model is improved.
3. The remote sensing target detection method based on the weight dictionary has good universality, and can be used for improving other remote sensing target detection models based on deep learning and improving the small sample learning capability of the remote sensing target detection models.
Drawings
FIG. 1 is a flow chart of a small sample remote sensing target detection method based on weight dictionary learning according to the present invention;
fig. 2 is a schematic diagram of a training process in a small sample remote sensing target detection method based on weight dictionary learning according to an embodiment of the present application;
fig. 3 is a schematic view of a data set partitioning process of small sample remote sensing target detection based on weight dictionary learning according to an embodiment of the present application;
fig. 4 is a schematic diagram of a small sample remote sensing target detection framework based on weight dictionary learning according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a dictionary learning principle provided in an embodiment of the present application;
fig. 6 is a schematic diagram of a small sample remote sensing target detection system based on weight dictionary learning provided by the invention.
Detailed Description
For a better understanding of the present invention, reference is made to the following description taken in conjunction with the accompanying drawings and examples.
Example 1:
the invention provides a small sample remote sensing target detection method based on weight dictionary learning, which comprises the following steps of:
acquiring remote sensing image data to be classified;
bringing the data into a pre-trained target detection model to obtain a target class corresponding to the remote sensing image;
the target detection model is obtained by learning and training small sample data based on a weight dictionary.
Here, the training of the target detection model is shown in fig. 2, and includes:
(1) constructing a target detection data set based on historical remote sensing image data with target categories;
(2) dividing the remote sensing image target detection data set into a source class data set and a target class data set;
(3) training by using the source data set to obtain a single-stage target detection model, constructing a parameter dictionary based on convolutional layer parameters of the single-stage target model, setting a weight for each parameter in the parameter dictionary as a corresponding dictionary coefficient, and constructing a target detection model based on the weight dictionary based on the parameter dictionary and the corresponding dictionary coefficient;
(4) and training the target detection model based on the dictionary by using the target class data set to obtain an optimal target detection model.
Dividing the remote sensing image target detection data set into a source class data set and a target class data set, as shown in fig. 3, specifically including:
step S1: dividing target classes in a remote sensing image target detection data set into a source class and a target class;
step S2: screening the remote sensing image according to the target category contained in the remote sensing image: dividing a remote sensing image only containing a source type target into a source data set; dividing a remote sensing image only containing a target type target into a target data set; discarding the remote sensing image of the target simultaneously containing the source class and the target class from the data set, and ensuring that the target classes and data in the source class data set and the target class data set are different; here, the remote sensing ground object target categories include but are not limited to: airplanes, vehicles, ships, oil tanks, sewage treatment plants, basketball courts, football fields, tennis courts, airports, train stations, bridges, ports, overpasses, intersections, and the like.
Step S3: for the residual remote sensing images in the data set, dividing the images only containing the source type targets into a source data set, and dividing the images only containing the target type targets into a target data set;
(3) training by using the source data set to obtain a single-stage target detection model, constructing a parameter dictionary based on convolutional layer parameters of the single-stage target model, setting a weight for each parameter in the parameter dictionary as a corresponding dictionary coefficient, and constructing a dictionary-based target detection model based on the parameter dictionary and the corresponding dictionary coefficient.
In this embodiment, a target detection model is trained on a training set of a source data set, the target detection model is composed of a feature extractor and a single-stage target detector, and all network layers are convolutional layers. The training process is the same as a standard deep learning target detection model, and the model is trained using all samples in the training set until the model achieves the best performance on the test set of the source data set.
As shown in fig. 4, the method specifically includes:
step S1: dividing a source data set into a training set and a test set, wherein the training set and the test set contain CsourceTarget class, i-th target class in training set of source data set
Figure BDA0002551280770000081
At least comprises
Figure BDA0002551280770000082
(in general, provided with
Figure BDA0002551280770000083
) The number of training samples, i.e. the whole training set of the source data set, is
Figure BDA0002551280770000084
Step S2: the ith target class in the test set of the source data set
Figure BDA0002551280770000085
At least comprises
Figure BDA0002551280770000086
(in general, provided with
Figure BDA0002551280770000087
) Number of test samples, i.e. test set samples of the entire source data set
Figure BDA0002551280770000088
Step S3: for each input image, the target detection model firstly detects the positions of all targets, then classifies each target into one category in the data set, and if the image contains the targets of the airplane and ship categories, the target detection model detects the position of each target, and then classifies the target into the airplane or ship. On the source data set, use
Figure BDA0002551280770000089
Training a single-stage target detection model by sufficient training samples until the training samples are obtained
Figure BDA00025512807700000810
The best test performance was achieved on the sample of each test set, and then the model D was usedsAll the convolution layer parameters phi except the layer finally used for determining the object type and position are used as a parameter dictionary;
(4) training the dictionary-based target detection model by using the target class data set to obtain an optimal target detection model, as shown in fig. 5, specifically including:
step S1: constructing a dictionary-based object detection model D using a parametric dictionary of phi and corresponding dictionary coefficients wdWherein the parameter dictionary phi is fixed, only the dictionary coefficients w and DdThe last classification, regression layer parameter theta in (d) may be modified and the parameter quantity of the dictionary coefficient w is much smaller than the parameter quantity of the parameter dictionary (Num (w) < Num (phi)). Thus, compare with the original model DsOf (D), learnable parameters [ phi, theta ], model DdLess learnable parameter { w, theta } quantity of (D), i.e., Num (D)d)＜＜Num(Ds) I.e. model D based on weight dictionary learningdThe method is a lightweight target detection model. The parameter dictionary phi is trained on a remote sensing target detection task on source data, so that the parameter dictionary phi contains rich remote sensing field knowledge. In addition, when the remote sensing target detection samples in the source data set are limited, the parameter dictionary can be trained on the remote sensing image ground object classification data to ensure that the parameter dictionary has knowledge in the remote sensing field.
Parameters in the parameter dictionary φ:
the parameters in the parameter dictionary phi are composed of all the convolutional layer parameters therein.
In the parameter dictionary containing L convolutional layers, the L ∈ [1, L ] th]) Each convolutional layer is:
Figure BDA00025512807700000910
where l represents the number of layers and s represents the convolutional layer trained on the source data set. The parameters of each convolutional layer in the parameter dictionary phi are tensors with the shape of C × N × k × k, where C represents the number of convolutional layer output channels, N represents the number of convolutional layer input channels, and k represents the size of the convolutional kernel. Convolutional layer Conv with a shape of CxNxk x k in the parameter dictionary phisFor dictionary, a new convolutional layer can be constructed, the new convolutional layer ConvdShape of (2) and the original winding layer ConvsAre identical in shape. Will ConvsThis tensor, shaped as C × N × k × k, is decomposed into C sub-tensors, shaped as N × k × k, the C-th sub-tensor is denoted as Ts c. Conv for new convolution layerdSimilarly, the data can be decomposed into C sub-tensors with the shape of N × k × k, and the C-th sub-tensor is denoted as Td c. Each sub-tensor T of the new convolutional layerdConv from the original winding layersThe linear combination of all the sub-tensors in (a):
Figure BDA0002551280770000091
wherein, wcThe dictionary coefficients representing the corresponding c-th sub-tensor are also the weights. Using all new convolutional layers and adding convolutional layers for predicting target boundary regression after that
Figure BDA0002551280770000092
And convolutional layer for predicting target class
Figure BDA0002551280770000093
Constructing a target detection model D based on a dictionaryd. In conclusion, the new buildup layer ConvdThe construction process of (A) is as follows:
Figure BDA0002551280770000094
Figure BDA0002551280770000095
conv indicating a new convolutional layerdThe ith convolution kernel corresponds to the convolution layer Conv in the parameter dictionarysDictionary coefficients of the jth convolution kernel.
Step S2: on the target data set, C is contained togethertargetA target sample and a target class C in the source domain data setsourceIn a different way, i.e.
Figure BDA0002551280770000096
The ith target class in the training set
Figure BDA0002551280770000097
At most comprises
Figure BDA0002551280770000098
(in general, provided with
Figure BDA0002551280770000099
) The number of training samples, i.e. the whole training set of the source data set, is
Figure BDA0002551280770000101
) (ii) a Therefore, for the model, only dictionary parameters and convolution layers for regression and classification are available
Figure BDA0002551280770000102
And
Figure BDA0002551280770000103
thereby reducing the number of parameters participating in training.
Step S3: the ith object class in a test set of an object data set
Figure BDA0002551280770000104
At least comprises
Figure BDA0002551280770000105
(in general, provided with
Figure BDA0002551280770000106
) The number of test sample, i.e. test set sample of the target data set, is
Figure BDA0002551280770000107
)；
Step S4: in the training set of the target data set, only use
Figure BDA0002551280770000108
Training a dictionary-based remote sensing target detection model by using a small number of training samples, and optimizing dictionary coefficients w and DdThe final classification and regression layer parameter theta in the process of the remote sensing target detection model D under the condition of small samplesdThe objective function of the dictionary coefficient optimization is as follows:
Figure BDA0002551280770000109
where w represents dictionary coefficients and θ represents model DdConvolution layer for regression and classification
Figure BDA00025512807700001010
And
Figure BDA00025512807700001011
i denotes the input image,
Figure BDA00025512807700001012
and
Figure BDA00025512807700001013
respectively identify the tag and the location tag. The parameter dictionary contains rich remote sensing field knowledge, so that the model constructed based on the parameter dictionary can effectively realize small sample detection of the new-class remote sensing target.
Based on the above-described objective function, a dictionary-based object detection model D is trained on a small number of samples on a training set in a target datasetdOptimizing dictionary coefficient w, regression and classification layer parameters theta, and then testing on a test set of a target data set, thereby completing the remote sensing image target detection model D under the condition of small samplesdTraining and testing.
In addition, considering that the number of samples in the training set of the target data set is small and not representative, in order to make the test result more reliable, the target data set is generally divided into M times repeatedly, and then the model D is performed separatelydAnd finally, taking the average value of the test results in the M divisions as a final test result.
In the case of the example 2, the following examples are given,
in order to implement the method, the present invention further provides a small sample remote sensing target detection system based on weight dictionary learning, as shown in fig. 6, including:
the data acquisition module is used for acquiring remote sensing image data to be classified;
the target detection module is used for substituting the data into a target detection model which is trained by the target detection model building module in advance to obtain the position and the category of the remote sensing target in the remote sensing image;
and the target detection model construction module is used for performing learning training on the basis of the dictionary by using the small sample data to obtain a target detection model.
The target detection model building module comprises:
the target detection data set construction unit is used for constructing a target detection data set based on the historical remote sensing image data with the target category;
the target detection data set dividing unit is used for dividing the remote sensing image target detection data set into a source data set and a target data set;
the target detection model establishing unit is used for training by utilizing the source data set to obtain a single-stage target detection model, establishing a parameter dictionary based on the convolutional layer parameters of the single-stage target model, setting a corresponding dictionary coefficient for each parameter in the parameter dictionary, and establishing a dictionary-based target detection model based on the parameter dictionary and the corresponding dictionary coefficient; and is also used for: and training the target detection model based on the dictionary by using the target class data set to obtain an optimal target detection model.
The target detection data set dividing unit specifically includes:
dividing target classes in a remote sensing image target detection data set into a source class and a target class;
discarding the remote sensing image of the target simultaneously containing the source class and the target class from the data set, and ensuring that the target classes and data in the source class data set and the target class data set are different; here, the remote sensing ground object target categories include but are not limited to: airplanes, vehicles, ships, oil tanks, sewage treatment plants, basketball courts, football fields, tennis courts, airports, train stations, bridges, ports, overpasses, intersections, and the like.
For the residual remote sensing images in the data set, dividing the images only containing the source type targets into a source data set, and dividing the images only containing the target type targets into a target data set;
the target detection model establishing unit specifically comprises:
dividing a source data set into a training set and a test set, wherein the training set and the test set contain CsourceTarget class, i-th target class in training set of source data set
Figure BDA0002551280770000111
At least comprises
Figure BDA0002551280770000112
(in general, provided with
Figure BDA0002551280770000113
) The number of training samples, i.e. the whole training set of the source data set, is
Figure BDA0002551280770000114
The ith target class in the test set of the source data set
Figure BDA0002551280770000115
At least comprises
Figure BDA0002551280770000116
(in general, provided with
Figure BDA0002551280770000117
) Number of test samples, i.e. test set samples of the entire source data set
Figure BDA0002551280770000118
For each input image, the target detection model firstly detects the positions of all targets, then classifies each target into one category in the data set, and if the image contains the targets of the airplane and ship categories, the target detection model detects the position of each target, and then classifies the target into the airplane or ship. On the source data set, use
Figure BDA0002551280770000121
Training a single-stage target detection model by sufficient training samples until the training samples are obtained
Figure BDA0002551280770000122
The best test performance was achieved on the sample of each test set, and then the model D was usedsAll the convolution layer parameters phi except the layer finally used for determining the object type and position are used as a parameter dictionary;
constructing a dictionary-based object detection model D using a parametric dictionary of phi and corresponding dictionary coefficients wdWherein the parameter dictionary phi is fixed, only the dictionary coefficients w and DdThe last classification, regression layer parameter theta in (d) may be modified and the parameter quantity of the dictionary coefficient w is much smaller than the parameter quantity of the parameter dictionary (Num (w) < Num (phi)). Thus, compare with the original model DsOf (D), learnable parameters [ phi, theta ], model DdLess learnable parameter { w, theta } quantity of (D), i.e., Num (D)d)＜＜Num(Ds) I.e. model D based on weight dictionary learningdThe method is a lightweight target detection model. The parameter dictionary phi is trained on a remote sensing target detection task on source data, so that the parameter dictionary phi contains rich remote sensing field knowledge. In addition, when the remote sensing target detection samples in the source data set are limited, the parameter dictionary can be trained on the remote sensing image ground object classification data to ensure that the parameter dictionary has knowledge in the remote sensing field.
Parameters in the parameter dictionary φ:
the parameters in the parameter dictionary phi are composed of all the convolutional layer parameters therein.
The parameters of each convolutional layer in the parameter dictionary phi are tensors with the shape of C × N × k × k, where C represents the number of convolutional layer output channels, N represents the number of convolutional layer input channels, and k represents the size of the convolutional kernel. Convolutional layer Conv with a shape of CxNxk x k in the parameter dictionary phisFor dictionary, a new convolutional layer can be constructed, the new convolutional layer ConvdShape of (2) and the original winding layer ConvsAre identical in shape. Will ConvsThis tensor, shaped as C × N × k × k, is decomposed into C sub-tensors, shaped as N × k × k, the C-th sub-tensor is denoted as Ts c. Conv for new convolution layerdSimilarly, the data can be decomposed into C sub-tensors with the shape of N × k × k, and the C-th sub-tensor is denoted as Td c. Each sub-tensor T of the new convolutional layerdConv from the original winding layersThe linear combination of all the sub-tensors in (a):
Figure BDA0002551280770000131
wherein, wcDictionary coefficients representing the corresponding c-th sub-tensor. In conclusion, the new buildup layer ConvdThe construction process of (A) is as follows:
Figure BDA0002551280770000132
Figure BDA0002551280770000133
conv indicating a new convolutional layerdThe ith convolution kernel corresponds to the convolution layer Conv in the parameter dictionarysDictionary coefficients of the jth convolution kernel.
On the target data set, C is contained togethertargetA target sample and a target class C in the source domain data setsourceIn a different way, i.e.
Figure BDA0002551280770000134
The ith target class in the training set
Figure BDA0002551280770000135
At most comprises
Figure BDA0002551280770000136
(in general, provided with
Figure BDA0002551280770000137
) The number of training samples, i.e. the whole training set of the source data set, is
Figure BDA0002551280770000138
The ith object class in a test set of an object data set
Figure BDA0002551280770000139
At least comprises
Figure BDA00025512807700001310
(in general, provided with
Figure BDA00025512807700001311
) The number of test sample, i.e. test set sample of the target data set, is
Figure BDA00025512807700001312
Step S4: in the training set of the target data set, only use
Figure BDA00025512807700001313
Training a dictionary-based remote sensing target detection model by using a small number of training samples, and optimizing dictionary coefficients w and DdThe final classification and regression layer parameter theta in the process of the remote sensing target detection model D under the condition of small samplesdThe objective function of the dictionary coefficient optimization is as follows:
Figure BDA00025512807700001314
where w represents dictionary coefficients and θ represents model DdConvolution layer for regression and classification
Figure BDA00025512807700001315
And
Figure BDA00025512807700001316
i denotes the input image,
Figure BDA00025512807700001317
and
Figure BDA00025512807700001318
respectively identify the tag and the location tag. The parameter dictionary contains rich remote sensing field knowledge, so that the model constructed based on the parameter dictionary can effectively realize small sample detection of the new-class remote sensing target.
In addition, considering that the target data set has a smaller number of samples in the training set, which is not representative, the target data set is generally duplicated to make the test result more reliableDividing M times, and performing model DdAnd finally, taking the average value of the test results in the M divisions as a final test result.

Claims (11)

1. A small sample remote sensing target detection method based on weight dictionary learning is characterized by comprising the following steps:
acquiring remote sensing image data to be classified;
bringing the data into a pre-trained target detection model to obtain a target class corresponding to the remote sensing image;
the target detection model is obtained by learning and training small sample data based on a weight dictionary;
the training of the target detection model comprises:
constructing a target detection data set based on historical remote sensing image data with target categories;
dividing the remote sensing image target detection data set into a source data set and a target data set;
training by using the source data set to obtain a single-stage target detection model, constructing a parameter dictionary based on convolutional layer parameters of the single-stage target detection model, setting a corresponding dictionary coefficient for each parameter in the parameter dictionary, and constructing a target detection model based on a weight dictionary based on the parameter dictionary and the corresponding dictionary coefficient;
and training the target detection model based on the weight dictionary by using the target data set to obtain an optimal target detection model.
2. The method for detecting the target of claim 1, wherein the dividing the set of remotely sensed image target detection data into a source data set and a target data set comprises:
dividing the target class in the remote sensing image target detection data set into a source class target and a target class target;
discarding the remote sensing image simultaneously containing the source type target and the target type target from the data set;
and for the residual remote sensing images in the data set, dividing the images only containing the source type targets into a source data set, and dividing the images only containing the target type targets into a target data set.
3. The method of claim 1, wherein the training with the source data set to obtain a single-stage target detection model, and constructing a parameter dictionary based on convolutional layer parameters of the single-stage target detection model, setting a corresponding dictionary coefficient for each parameter in the parameter dictionary, and constructing a target detection model based on a weight dictionary based on the parameter dictionary and the corresponding dictionary coefficient comprises:
dividing the source data set into a training set and a test set;
model D for single-stage target detection by using samples in training setsTraining, and testing by using the samples in the test set until the best test performance is achieved on all the samples in the test set;
then detecting the model D by the single-stage targetsAll the convolution layer parameters phi except the layer finally used for determining the object type and position are used as a parameter dictionary;
setting a corresponding dictionary coefficient w for each dictionary parameter in the parameter dictionary phi; wherein the initial value of the dictionary coefficient is randomly determined;
constructing a dictionary-based object detection model D using a parameter dictionary consisting of all convolutional layer parameters phi and corresponding dictionary coefficients wd；
Wherein the parameter dictionary phi is fixed, dictionary coefficients w and the target detection model DdThe determined classification, regression layer parameters theta may be modified and the parameter quantities of the dictionary coefficients w are much smaller than the parameter quantities of the parameter dictionary.
4. The object detection method according to claim 3, wherein the parameters in the parameter dictionary Φ are constituted by parameters of all convolution layers;
the parameters of each convolutional layer in the parameter dictionary phi are tensors of the shape C × N × k × k.
5. The object detection method of claim 4, wherein a dictionary-based object detection model D is constructed using a parameter dictionary consisting of all convolutional layer parameters φ and corresponding dictionary coefficients wdThe method comprises the following steps:
initial convolutional layer Conv with a shape of CxNxk x k in the parameter dictionary phisIs a dictionary;
conv of the initial convolution layersDecomposing into C sub-tensors of shape Nxk x k;
conv for initial convolutional layersLinearly combining all the sub-tensors to form each sub-tensor T in the target convolutional layerdTaking each sub tensor as a convolution kernel, and establishing a dictionary coefficient for each convolution kernel;
based on the sub-tensor TdConstruction of a target convolutional layer Conv with corresponding dictionary coefficientsdWherein the target convolutional layer ConvdShape of (2) and initial convolution layer ConvsAre the same in shape;
conv from the target convolutional layerdConstructing an object detection model Dd。
6. The target detection method of claim 5, wherein the target convolutional layer ConvdThe construction process of each dictionary coefficient is as follows:
Figure FDA0002914729390000031
in the formula (I), the compound is shown in the specification,
Figure FDA0002914729390000032
conv representing the target convolutional layerdThe ith convolution kernel corresponds to the convolution layer Conv in the parameter dictionarysDictionary coefficients of the jth convolution kernel;
Figure FDA0002914729390000033
representation parameter dictionaryA set of all sub-tensors in the medium convolution layer;
Figure FDA0002914729390000034
a set representing all sub-tensors in the target convolutional layer; t iss c: representing the c-th sub-tensor in the convolutional layer in the parametric dictionary.
7. The object detection method of claim 6, wherein the sub-tensor TdThe expression of (a) is as follows:
Figure FDA0002914729390000035
wherein, wcDictionary coefficients representing the corresponding c-th sub-tensor.
8. The method of claim 1, wherein the training the weight dictionary-based object detection model using the set of object data to obtain an optimal object detection model comprises:
dividing a target training set and a target testing set on a target data set;
training a remote sensing target detection model based on a weight dictionary by using the samples of the target training set, and optimizing dictionary coefficients w and DdFinally, determining a parameter theta of the target category and the target position;
testing the optimized remote sensing target detection model based on the weight dictionary by using the samples in the target test set to determine a remote sensing target detection model D under the condition of small samplesd。
9. The object detection method of claim 1, wherein the dictionary coefficient optimized objective function is as follows:
Figure FDA0002914729390000041
wherein D isw，θ(I) Representing a small sample object detection model, w representing dictionary coefficients, and theta representing a model DdConvolution layer for regression and classification
Figure FDA0002914729390000042
And
Figure FDA0002914729390000043
i denotes the input image.
10. The method of claim 8, wherein the training the weight dictionary-based object detection model using the set of object data to obtain an optimal object detection model, further comprises:
dividing the target data set into a target training set and a target testing set for multiple times;
training the target detection model based on the weight dictionary aiming at the target training set and the target testing set which are divided each time;
and evaluating the test results in multiple training, and taking the average value of the evaluation as the final test evaluation result.
11. A small sample remote sensing target detection system based on weight dictionary learning is characterized by comprising:
the data acquisition module is used for acquiring remote sensing image data to be classified;
the target detection module is used for bringing the data into a pre-trained target detection model to obtain the position and the category of the remote sensing target in the remote sensing image;
the target detection model is obtained by learning and training small sample data based on a weight dictionary;
the target detection model construction module is used for performing learning training on the small sample data based on the weight dictionary to obtain a target detection model;
the target detection model building module comprises:
the target detection data set construction unit is used for constructing a target detection data set based on the historical remote sensing image data with the target category;
the target detection data set dividing unit is used for dividing the remote sensing image target detection data set into a source data set and a target data set;
the target detection model establishing unit is used for training by utilizing the source data set to obtain a single-stage target detection model, establishing a parameter dictionary based on the convolutional layer parameters of the single-stage target detection model, setting a corresponding dictionary coefficient for each parameter in the parameter dictionary, and establishing a dictionary-based target detection model based on the parameter dictionary and the corresponding dictionary coefficient; and is also used for: and training the target detection model based on the weight dictionary by using the target data set to obtain an optimal target detection model.
CN202010576615.4A 2020-06-22 2020-06-22 Small sample remote sensing target detection method and system based on weight dictionary learning Active CN111860178B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010576615.4A CN111860178B (en) 2020-06-22 2020-06-22 Small sample remote sensing target detection method and system based on weight dictionary learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010576615.4A CN111860178B (en) 2020-06-22 2020-06-22 Small sample remote sensing target detection method and system based on weight dictionary learning

Publications (2)

Publication Number Publication Date
CN111860178A CN111860178A (en) 2020-10-30
CN111860178B true CN111860178B (en) 2021-03-23

Family

ID=72988378

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010576615.4A Active CN111860178B (en) 2020-06-22 2020-06-22 Small sample remote sensing target detection method and system based on weight dictionary learning

Country Status (1)

Country Link
CN (1) CN111860178B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112686289A (en) * 2020-12-24 2021-04-20 微梦创科网络科技（中国）有限公司 Picture classification method and device
CN116912630A (en) * 2023-09-12 2023-10-20 深圳须弥云图空间科技有限公司 Target identification method and device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103258210A (en) * 2013-05-27 2013-08-21 中山大学 High-definition image classification method based on dictionary learning
CN103761531A (en) * 2014-01-20 2014-04-30 西安理工大学 Sparse-coding license plate character recognition method based on shape and contour features
CN104680182A (en) * 2015-03-09 2015-06-03 西安电子科技大学 Polarimetric SAR classification method on basis of NSCT and discriminative dictionary learning
CN110414616A (en) * 2019-08-02 2019-11-05 南京大学 A kind of remote sensing images dictionary learning classification method using spatial relationship

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111126287B (en) * 2019-12-25 2022-06-03 武汉大学 Remote sensing image dense target deep learning detection method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103258210A (en) * 2013-05-27 2013-08-21 中山大学 High-definition image classification method based on dictionary learning
CN103761531A (en) * 2014-01-20 2014-04-30 西安理工大学 Sparse-coding license plate character recognition method based on shape and contour features
CN104680182A (en) * 2015-03-09 2015-06-03 西安电子科技大学 Polarimetric SAR classification method on basis of NSCT and discriminative dictionary learning
CN110414616A (en) * 2019-08-02 2019-11-05 南京大学 A kind of remote sensing images dictionary learning classification method using spatial relationship

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Discriminative K-SVD for dictionary learning in face recognition;Zhang Q;《IEEE Conference on Computer Vision and Pattern Recognition（CVPR）》;20101031;全文 *
基于双权重约束的判别字典学习算法;李争名;《计算机与数字工程》;20170531;第45卷(第5期);全文 *

Also Published As

Publication number Publication date
CN111860178A (en) 2020-10-30

Similar Documents

Publication Publication Date Title
CN111368896B (en) Hyperspectral remote sensing image classification method based on dense residual three-dimensional convolutional neural network
CN111739075B (en) Deep network lung texture recognition method combining multi-scale attention
CN108095716B (en) Electrocardiosignal detection method based on confidence rule base and deep neural network
EP3620990A1 (en) Capturing network dynamics using dynamic graph representation learning
CN107292330B (en) Iterative label noise identification algorithm based on double information of supervised learning and semi-supervised learning
CN109117883B (en) SAR image sea ice classification method and system based on long-time memory network
CN106295714A (en) A kind of multi-source Remote-sensing Image Fusion based on degree of depth study
CN111860178B (en) Small sample remote sensing target detection method and system based on weight dictionary learning
CN111090764B (en) Image classification method and device based on multitask learning and graph convolution neural network
CN106408030A (en) SAR image classification method based on middle lamella semantic attribute and convolution neural network
CN111860106B (en) Unsupervised bridge crack identification method
Wang et al. A computer vision based machine learning approach for fatigue crack initiation sites recognition
Joutsensaari et al. Identification of new particle formation events with deep learning
CN110082738B (en) Radar target identification method based on Gaussian mixture and tensor recurrent neural network
Savino et al. Automated classification of civil structure defects based on convolutional neural network
CN111798417A (en) SSD-based remote sensing image target detection method and device
CN111709318A (en) High-resolution remote sensing image classification method based on generation countermeasure network
CN111861909A (en) Network fine-grained image denoising and classifying method
CN116977710A (en) Remote sensing image long tail distribution target semi-supervised detection method
CN116151319A (en) Method and device for searching neural network integration model and electronic equipment
Chou et al. SHM data anomaly classification using machine learning strategies: A comparative study
CN117152503A (en) Remote sensing image cross-domain small sample classification method based on false tag uncertainty perception
CN113723572B (en) Ship target identification method, computer system, program product and storage medium
CN113095229B (en) Self-adaptive pedestrian re-identification system and method for unsupervised domain
Liu et al. A novel deep framework for change detection of multi-source heterogeneous images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant