CN114169442A - Remote sensing image small sample scene classification method based on double prototype network - Google Patents
Remote sensing image small sample scene classification method based on double prototype network Download PDFInfo
- Publication number
- CN114169442A CN114169442A CN202111495585.5A CN202111495585A CN114169442A CN 114169442 A CN114169442 A CN 114169442A CN 202111495585 A CN202111495585 A CN 202111495585A CN 114169442 A CN114169442 A CN 114169442A
- Authority
- CN
- China
- Prior art keywords
- sample
- prototype
- query
- loss
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000008569 process Effects 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 4
- 238000005259 measurement Methods 0.000 claims description 4
- 239000000470 constituent Substances 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000000638 solvent extraction Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 2
- 238000007635 classification algorithm Methods 0.000 abstract description 7
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 238000002474 experimental method Methods 0.000 abstract description 2
- 230000003993 interaction Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 9
- 230000000694 effects Effects 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 230000003321 amplification Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- SHGAZHPCJJPHSC-NUEINMDLSA-N Isotretinoin Chemical compound OC(=O)C=C(C)/C=C/C=C(C)C=CC1=C(C)CCCC1(C)C SHGAZHPCJJPHSC-NUEINMDLSA-N 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 229960005280 isotretinoin Drugs 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a remote sensing image small sample scene classification algorithm based on a double prototype network. Firstly, the invention utilizes the supervision information of the support label to carry out prototype self-calibration on the prototype supporting the feature generation, and obtains a more accurate prototype. And then, on the basis of the calibrated accurate prototype, converting the prediction result of the query sample to be used as the query prototype, and reversely predicting the support sample by using the query prototype. In this process, the information interaction between the support sample and the query sample is a further calibration of the prototype, which is called prototype mutual calibration. The method optimizes three losses, wherein the self-calibration loss and the mutual-calibration loss are beneficial to learning more representative prototypes by the model and making more accurate prediction. The model of the invention is very lightweight and easy to use, since the network does not need to learn additional parameters. Experiments on three public remote sensing data sets show that the method has better classification performance compared with other advanced small sample image classification methods.
Description
Technical Field
The invention belongs to the field of remote sensing image recognition, and particularly relates to a remote sensing image small sample scene classification method based on a double prototype network.
Background
Remote sensing image scene classification has wide application in the real world, such as natural disaster detection, land utilization classification, city planning, etc. In recent years, deep learning has become a powerful means of remote sensing scene classification. However, two basic problems remain with the scene classification of remote sensing images. First, although it can predict the test samples of scene classes seen in the training phase with a high accuracy, it may be a dilemma in the face of class samples never seen in the training phase. Second, it is well known that the training process of the conventional deep learning method requires a large amount of labeled data, which is difficult to obtain in a new unknown environment. Under the background, the research on the small sample scene classification algorithm of the remote sensing image is widely concerned in the last two years. The purpose of small sample learning is to learn new concepts from a very small number of samples.
The existing small sample image classification method can be broadly divided into four categories, namely a small sample image classification algorithm based on a model, a small sample image classification algorithm based on measurement, a small sample image classification algorithm based on optimization and a small sample image classification algorithm based on data amplification. The model-based method aims to quickly update parameters of a small number of samples by designing a model structure and directly establish a mapping function between input and prediction, but the traditional gradient descent algorithm has more parameters and cannot quickly realize optimization. Metric-based methods mainly learn the mapping from the image to the embedding space and make the space some distinctiveness. The purpose of the small sample image classification algorithm based on optimization is to obtain a better initialized model or gradient descending direction, so that the model still has good generalization capability when facing a new class with limited sample size, however, the method is easy to be trapped in a local optimal point due to limited data size. The method based on data amplification proposes to generate false data by using a small amount of marker samples so as to realize data amplification, but noise is easily brought to a network due to irrational generated data.
Among the above small sample image classification methods, the metric-based method has been most widely studied due to its simple, effective, and easily reproducible characteristics. Among them, the prototype Network (Prototypical Network) is one of the most classical small sample classification methods based on metrics, and is described in documents "j.snell, k.swersky, and r.zemel.prototypical Networks for raw-shot learning.in.proc.adv.neural in.process.syst., 2017, pp.4077-4087.
Researchers have also made many extensions and improvements based on prototype networks. Some supplement prototypes by adding a priori knowledge (such as semantic word vectors and the like) in a training stage, and some add an attention-like mechanism in the whole network to improve the feature extraction capability of the network. However, these methods bring many extra calculations, and burden the network, and their usability in practical application scenarios is still considered regardless of the accuracy improvement effect. Therefore, for the classification of small samples of remote sensing images, how to use an algorithm which is as "economic" as possible to improve the classification accuracy is a long-term development direction in the future.
Disclosure of Invention
The invention provides a small sample classification method, namely a double Prototype Network (SPNet), which can effectively improve the Prototype effect and reduce the pressure of the Network. Specifically, in the case that the structure of the network only comprises a backbone network (ResNet-18), another prototype, namely a 'query prototype', is calculated from the existing information, and the prototype is combined with the original prototype and called as a 'double prototype', and two operations of 'prototype self-calibration' and 'prototype mutual calibration' are designed, so that the prototype is more representative in the training process and is more beneficial to the subsequent classification prediction based on the prototype.
The technical scheme adopted by the invention is as follows:
a remote sensing image small sample scene classification method based on a double prototype network comprises the following steps:
(1) preparing data: dividing a remote sensing image data set into a base class data set and a new class data set, carrying out data preprocessing, and dividing a preprocessed base class data set image into a support set and a query set;
(2) feature extraction: randomly selecting a support set sample and a query set sample from a base class data set according to tasks, respectively sending the support set sample and the query set sample into a ResNet-18 network for feature extraction, respectively obtaining a support sample feature and a query sample feature, and mapping the features into a dimension space of n multiplied by 1 by adopting global average pooling operation; wherein n represents the number of channels;
(3) respectively calculating an original prototype and a query prototype according to the support sample and the query sample, calculating original loss based on the characteristics of the original prototype and the query sample, calculating self-calibration loss based on the characteristics of the original prototype and the support sample, calculating mutual calibration loss based on the original prototype and the query prototype, weighting to obtain a total loss value, adjusting ResNet-18 network parameters according to the total loss value, and returning to the step (2) until the loss function is converged;
(4) and optimizing the trained ResNet-18 network by using the labeled samples in the new data set, extracting the characteristics of the remote sensing image based on the optimized ResNet-18 network, and classifying the remote sensing image by adopting a mode of combining cosine measurement and a softmax classifier.
Further, the step (1) of dividing the remote sensing image dataset into a base class dataset and a new class dataset comprises the following steps:
partitioning a remote sensing image dataset D into base class datasets DbaseAnd a new class data set DnovelBase class data set DbaseAnd a new class data set DnovelThere is no coincidence in the categories in (1); base class data set DbaseThe images in (1) are provided with labels and used for training the network; new class data set DnovelOnly k labeled images in each category are used for adjusting the ResNet-18 network, and the rest unlabeled images are used for testing the network performance; where k is a set value.
Further, the step (3) calculates the original loss, the self-calibration loss and the mutual-calibration loss of the support sample feature and the query sample feature, and weights the original loss, the self-calibration loss and the mutual-calibration loss to obtain an overall loss value, and the specific process includes:
calculating the raw loss Lori: first, the original prototype P of each class c is calculatedc：
Wherein x isiIs to support RGB image in set S, fθ(. for) a ResNet-18 network, ScRepresenting a subset of all sample constituents supporting class c in the set, K representing ScThe number of samples in (1); then, the original prototype P is utilizedcAnd query sample features fθ(qi) Performing a cosine measure d (-) to obtain a query sample qiProbability of belonging to a certain class c
Then, the cross entropy loss calculation is used to obtain the original loss Lori：
Where c represents the query sample qiCorresponding label, NqRepresenting the number of query samples in each task; beta is a scaling factor, so that cosine measurement can be more remarkable in a Softmax function, and the score of each class is easier to distinguish; c is the target category number of the support set in each task, and C' is any target label in the support set of each task;
calculating self-calibration loss Lself: using the original prototype PcAnd supporting sample features fθ(xi) Performing a cosine measure d (-) to obtain a support sample xiProbability of belonging to a certain class c
The loss predicted by the metric between the original prototype and the supporting sample features is the self-calibration loss, and the calculation method is as follows:
calculating the mutual calibration loss Linter: generating query prototypes from the query samples, using the query prototypes
And supporting sample features fθ(xi) Performing a cosine measure d (-) to obtain a support sample xiProbability of belonging to a certain class c
And mutually calibrating with the original prototype to calculate the mutual calibration loss Linter：
Wherein,
a query prototype representing class c is shown,
representing the number of samples in query set Q that belong to class c,
a sample representing class c in the query set;
calculating the total loss: l ═ Lori+λ1Lself+λ2Linter(ii) a Wherein λ is1And λ2Representing the weights of self-calibration loss and mutual-calibration loss, respectively.
Compared with the prior art, the invention adopting the technical scheme has the following technical effects:
according to the remote sensing image small sample scene classification method based on the double prototype network, the class prototypes are more representative in the training process through two operations of 'prototype self-calibration' and 'prototype mutual calibration', so that the classification prediction based on the class prototypes is more beneficial to follow-up classification prediction. Meanwhile, as no additional module or attention mechanism exists in the whole network and no complex learnable parameters exist, the method is remarkable in light weight and rapid calculation. The experimental results on the three remote sensing image reference data sets prove that the performance of the double prototype network is superior to that of other methods.
Drawings
FIG. 1 is a schematic diagram of the technical idea of the present invention;
FIG. 2 is an overall frame diagram (taking 5-way 5-shot as an example) of the remote sensing image small sample scene classification method based on the double prototype network.
Detailed Description
The present invention will be further described with reference to the following drawings and examples, which include, but are not limited to, the following examples.
As shown in fig. 1, the innovation of the network is that the information flow and information richness in the whole training process are changed greatly. The "original prototype self-calibration" is to calibrate an "original prototype" by using the characteristics of a support sample, and the "original prototype" is calculated by the support sample and is called as the "self-calibration"; and "prototype mutual calibration" is a calibration between "query prototype" and "original prototype". Both of the calibrations are finally blended into the loss function in a constraint mode, so that the whole network has no additional module or mechanism and no complex learnable parameters, and the method is remarkable in light weight and rapid calculation.
The hardware environment for implementation of the embodiment is as follows: intel (R) core (TM) i3-9100 CPU computer, 8.0GB memory, the software environment of operation is: ubuntu16.04.5lts and Pycharm 2019. The data sets NWPU-RESISC45, WHU-RS19, and UC Merced were published using large-scale remote sensing images. The NWPU-rescisc 45 dataset contains 45 scene classes, each of which has 700 RGB images of 256 × 256, and in the present embodiment, 35 classes are divided as a base class dataset D _ base, where 25 classes are used for training, 10 classes are used for verification (validation), and the remaining 10 classes are used as a new class dataset D _ novel. The WHU-RS19 data set has 19 scene categories, 1005 RGB images of 600 × 600 in total, 9 categories are divided for training, 5 categories are used for verification, and the remaining 5 are used as new category data. UC Merced has 21 scene categories, 100 RGB images of 256 × 256 each, 10 categories for training, 6 for verification, and the remaining five as new categories for testing. Under the 5-way 1-shot setting, each task includes 5 categories, each of which picks 1 support image and 15 query images. Under the 5-way 5-shot setting, each task includes 5 categories, each of which picks 5 support images and 15 query images.
As shown in fig. 2, the specific implementation process of the present invention is as follows:
1. preparing data
Partitioning a remote sensing image dataset D into base class datasets DbaseAnd a new class data set Dnovel，DbaseAnd DnovelThere is no coincidence of the categories in (1). DbaseThe images in (1) are used to train the network; dnovelOnly k (1 or 5) labeled images in each category are used for optimizing the network, and the rest unlabeled images are used for testing the network performance. In the training process, images in the base class are sampled and preprocessed: each iteration process simulates the setting of small sample learning, namely 5 categories are randomly selected, each category only comprises k (1 or 5, respectively forming tasks of 5-way 1-shot and 5-way 5-shot) labeled samples for training (forming a support set S), and 15 unlabeled samples for verification (forming a query set Q). Each sample is then cropped to 224 x 224, then randomly horizontally flipped, brightness enhanced, color enhanced, contrast enhanced data enhanced, and normalized. Dividing the preprocessed base class data set image into a support set S and a query set Q;
the normalization treatment comprises the following steps:
normalizing the three RGB channels of each image I according to the following formula:
wherein, IhH channel, I, representing an imageh' denotes the h channel after normalization, MeanhRepresents the mean value of the h channel, StdhRepresents the standard deviation of the h channel.
Random horizontal flipping, brightness enhancement, color enhancement, contrast enhancement these four data enhancement operations are all implemented using torchvision.
2. Feature extraction
Randomly selecting a support set sample and a query set sample from the base class data set according to tasks, sending the support set sample and the query set sample into a ResNet-18 network for feature extraction, and respectively obtaining the support sample features fθ(xi) And query sample features fθ(qi). Wherein xi∈S,qi∈Q，fθ(. cndot.) represents a ResNet-18 network, and a global averaging pooling operation is added at the end of the network to map features into a dimension space of n × 1 × 1, with n representing the number of channels. This is added because of the requirement for dual prototype networks to compute prototypes, since prototypes are defined as n × 1 × 1 vectors that can represent class features, as will be further described below.
3. Calculating raw loss
First, the prototypical p is calculated by referring to the PrototypicalNetwork. E.g. original prototype P of class ccThe following were used:
wherein ScRepresenting a subset of all sample constituents supporting class c in the set, K representing ScIs detected (i.e., 1 or 5). The concept of prototype is the feature mean of the supporting sample, which can simply and effectively extract the part of the sample that can represent the most class features.
Using the original prototype PcAnd query features fθ(qi) Performing a cosine measure d (-) to obtain a query sample qiProbability of belonging to a certain class c
Then, the cross entropy loss calculation is used to obtain the original loss Lori：
The two formulas above use the Softmax classification function and the cross entropy loss function, respectively. Where c represents the query sample qiCorresponding label, NqRepresenting the number of query samples in each task, C being the number of target categories of a support set in each task, and C' being any type of target labels in the support set of each task; to make the cosine measure more prominent in the Softmax function and to make the score of each class more distinguishable, a scaling factor β, set to 50 in this example, is used to scale the cosine similarity to a range of [0, 50%]。
4. Prototype self-calibration
In the original prototype network explained above, the prototype is simply obtained from the support features and used directly to predict the query sample. We believe that the information flow in this process is not sufficient to produce an effective prototype. Therefore, we propose a prototype self-calibration mechanism to further exploit the information that supports the sample. In particular, by the original prototype PcAnd support sample xiThe support samples are predicted by the metric between, since the original prototype is calculated by the sample features in the support set, this prediction is called self-calibration, and similarly, the classification prediction method using the Softmax function is as follows:
further, we can direct the model to calibrate the prototype with the loss of self-calibration calculated by:
c and K are consistent with those of the C-way K-shot task, namely respectively representing the number of categories in the support set in each task and the number of samples of each category in the support set in each task.
5. Mutual calibration of prototypes
Then we continue to mine the information in the given sample. Based on the calibrated accurate prototype, a new method for predicting the support sample based on the information of the query sample is provided. Specifically, we use the predicted probability of each class of the query sample as another prototype of the class, query prototype PqAnd use of PqAs a guide, the support samples are backward predicted in the same way as the query samples in the original prototype network are predicted. Based on PqBy influencing the model's predictions of the query, the prototype is implicitly calibrated. More precisely, the effect between the two prototypes is mutual, i.e. the two prototypes interact in the process, while ensuring end-to-end training. Experimental results show that this interaction is positive and can force the model to achieve better performance. We call this calibration process a prototype cross-calibration, which can be expressed as:
wherein, the prototype P is queriedqThe calculation method of (2) is as follows:
wherein,
representing the number of samples in query set Q that belong to class c,
a sample representing class c in the query set,
representing a prototype of a class c query, which is formed from an average query sampleAll probabilities predicted to class c are calculated,
to make use of query prototypes
And supporting sample features fθ(xi) Performing a cosine measure d (-) to obtain a support sample xiProbability of belonging to a certain class c.
Therefore, cross entropy loss can be used to calculate the cross calibration loss:
c and K in the formula are consistent with those in the C-way K-shot task, namely representing the number of categories in the support set in each task and the number of samples of each category in the support set in each task respectively.
Finally, the overall loss function can be calculated:
L＝Lori+λ1Lself+λ2Linter
λ1and λ2Is a hyperparameter that adjusts the weight of the two auxiliary losses. In this example, it was determined by conducting a research experiment on the NWPU-RESISC45 dataset that the 5-way 1-shot scenario was set to 1 and 3, respectively, and the 5-way 5-shot scenario was set to 3 and 7, respectively.
6. Classification effect verification
By DnovelThe data set tests a ResNet-18 network trained with the above-described overall loss L. And (4) classifying by combining a cosine measure with a Softmax classifier. A total of two tasks, 5-way 1-shot and 5-way 5-shot, were tested, 600 randomly generated tasks in each case. In each task, firstly, the label samples in the new class data set are used for finely adjusting the trained ResNet-18 network, the finely adjusted ResNet-18 network is used for classifying the remote sensing images and calculating the average prediction accuracy as the final result.
And (4) evaluating the effectiveness of the method by selecting the classification accuracy rate accuray. The accure is the percentage of the correctly classified samples in the total samples, and generally, the larger the value of the accure is, the better the algorithm is. The accuray is calculated as follows:
where P and N represent positive and negative examples, respectively, TP represents a split positive example, and TN represents a split negative example.
The classification result obtained by the method is compared with the baseline method on the NWPU-RESISC45 data set, the comparison result is shown in Table 1, and the classification accuracy shows the effectiveness of the method. Compared with the method of the invention, the baseline model does not contain prototype self-calibration and prototype mutual-calibration operations. Specifically, the Baseline model is a Prototypical Network that takes the backbone Network as ResNet-18.
TABLE 1
| Model | 1-shot | 5-shot |
| Baseline | 65.20±0.84％ | 80.52±0.55％ |
| Ours | 67.84％±0.87％ | 83.94％±0.50％ |
And the classification result obtained by the method is compared with the baseline method on the WHU-RS19 data set and the UC Mercated data set, the comparison results are respectively shown in the table 2 and the table 3, the classification accuracy shows the effectiveness of the method on different data sets, and the method has superiority compared with the Prototypical Network with a backbone Network of ResNet-18.
TABLE 2
| Model | 1-shot | 5-shot |
| Baseline | 76.36％±0.67％ | 85.00％±0.36％ |
| Ours | 81.06％±0.60％ | 88.04％±0.28％ |
TABLE 3
| Model | 1-shot | 5-shot |
| Baseline | 53.85％±0.78％ | 71.23％±0.48％ |
| Ours | 57.64％±0.73％ | 73.52％±0.51％ |
Claims (3)
1. A remote sensing image small sample scene classification method based on a double prototype network is characterized by comprising the following steps:
(1) preparing data: dividing a remote sensing image data set into a base class data set and a new class data set, carrying out data preprocessing, and dividing a preprocessed base class data set image into a support set and a query set;
(2) feature extraction: randomly selecting a support set sample and a query set sample from a base class data set according to tasks, respectively sending the support set sample and the query set sample into a ResNet-18 network for feature extraction, respectively obtaining a support sample feature and a query sample feature, and mapping the features into a dimension space of n multiplied by 1 by adopting global average pooling operation; wherein n represents the number of channels;
(3) respectively calculating an original prototype and a query prototype according to the support sample and the query sample, calculating original loss based on the characteristics of the original prototype and the query sample, calculating self-calibration loss based on the characteristics of the original prototype and the support sample, calculating mutual calibration loss based on the original prototype and the query prototype, weighting to obtain a total loss value, adjusting ResNet-18 network parameters according to the total loss value, and returning to the step (2) until the loss function is converged;
(4) and optimizing the trained ResNet-18 network by using the labeled samples in the new data set, extracting the characteristics of the remote sensing image based on the optimized ResNet-18 network, and classifying the remote sensing image by adopting a mode of combining cosine measurement and a softmax classifier.
2. The remote sensing image small sample scene classification method based on the biprimary network as claimed in claim 1, wherein the step (1) of dividing the remote sensing image dataset into a base class dataset and a new class dataset comprises the following processes:
partitioning a remote sensing image dataset D into base class datasets DbaseAnd a new class data set DnovelBase class data set DbaseAnd a new class data set DnovelThere is no coincidence in the categories in (1); base class data set DbaseThe images in (1) are provided with labels and used for training the network; new class data set DnovelOnly k labeled images in each category are used for adjusting the ResNet-18 network, and the rest unlabeled images are used for testing the network performance; where k is a set value.
3. The remote sensing image small sample scene classification method based on the double prototype network as claimed in claim 1, wherein the original loss, the self calibration loss and the mutual calibration loss of the support sample feature and the query sample feature are calculated in the step (3), and weighting is performed to obtain the total loss value, and the specific process comprises:
calculating the raw loss Lori: first, the original prototype P of each class c is calculatedc：
Wherein x isiTo support RGB images in set S, fθ(. for) a ResNet-18 network, ScRepresenting a subset of all sample constituents supporting class c in the set, K representing ScThe number of samples in (1); then, the original prototype P is utilizedcAnd query sample features fθ(qi) Performing a cosine measure d (-) to obtain a query sample qiProbability of belonging to a certain class c
And then the original loss is obtained by using cross entropy loss calculationLose Lori：
Where c represents the query sample qiCorresponding label, NqRepresenting the number of query samples in each task, beta being a scaling factor; c is the target category number of the support set in each task, and C' is any target label in the support set of each task;
calculating self-calibration loss Lself: using the original prototype PcAnd supporting sample features fθ(xi) Performing a cosine measure d (-) to obtain a support sample xiProbability of belonging to a certain class c
The loss predicted by the metric between the original prototype and the supporting sample features is the self-calibration loss, and the calculation method is as follows:
calculating the mutual calibration loss Linter: generating query prototypes from the query samples, using the query prototypes
And supporting sample features fθ(xi) Performing a cosine measure d (-) to obtain a support sample xiProbability of belonging to a certain class c
And mutually calibrating with the original prototype to calculate the mutual calibration loss Linter：
Wherein,
a query prototype representing class c is shown,
representing the number of samples in query set Q that belong to class c,
a sample representing class c in the query set;
calculating the total loss: l ═ Lori+λ1Lself+λ2Linter(ii) a Wherein λ is1And λ2Representing the weights of self-calibration loss and mutual-calibration loss, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111495585.5A CN114169442B (en) | 2021-12-08 | 2021-12-08 | Remote sensing image small sample scene classification method based on double prototype network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111495585.5A CN114169442B (en) | 2021-12-08 | 2021-12-08 | Remote sensing image small sample scene classification method based on double prototype network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114169442A true CN114169442A (en) | 2022-03-11 |
| CN114169442B CN114169442B (en) | 2022-12-09 |
Family
ID=80484692
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111495585.5A Active CN114169442B (en) | 2021-12-08 | 2021-12-08 | Remote sensing image small sample scene classification method based on double prototype network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114169442B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114844583A (en) * | 2022-03-30 | 2022-08-02 | 电子科技大学 | End-to-end communication receiving method based on prototype network |
| CN114842236A (en) * | 2022-03-22 | 2022-08-02 | 西北工业大学 | Image classification method and device, computer-readable storage medium and electronic device |
| CN114936615A (en) * | 2022-07-25 | 2022-08-23 | 南京大数据集团有限公司 | Small sample log information anomaly detection method based on characterization consistency correction |
| CN115564960A (en) * | 2022-11-10 | 2023-01-03 | 南京码极客科技有限公司 | Network image label denoising method combining sample selection and label correction |
| CN115984621A (en) * | 2023-01-09 | 2023-04-18 | 宁波拾烨智能科技有限公司 | Small sample remote sensing image classification method based on restrictive prototype comparison network |
| CN116168257A (en) * | 2023-04-23 | 2023-05-26 | 安徽大学 | Small sample image classification method, device and storage medium based on sample generation |
| CN116778268A (en) * | 2023-04-20 | 2023-09-19 | 江苏济远医疗科技有限公司 | Sample selection deviation relieving method suitable for medical image target classification |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818903A (en) * | 2020-12-10 | 2021-05-18 | 北京航空航天大学 | Small sample remote sensing image target detection method based on meta-learning and cooperative attention |
| CN112861720A (en) * | 2021-02-08 | 2021-05-28 | 西北工业大学 | Remote sensing image small sample target detection method based on prototype convolutional neural network |
| CN113673599A (en) * | 2021-08-20 | 2021-11-19 | 大连海事大学 | Hyperspectral image classification method based on correction prototype learning |
-
2021
- 2021-12-08 CN CN202111495585.5A patent/CN114169442B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818903A (en) * | 2020-12-10 | 2021-05-18 | 北京航空航天大学 | Small sample remote sensing image target detection method based on meta-learning and cooperative attention |
| CN112861720A (en) * | 2021-02-08 | 2021-05-28 | 西北工业大学 | Remote sensing image small sample target detection method based on prototype convolutional neural network |
| CN113673599A (en) * | 2021-08-20 | 2021-11-19 | 大连海事大学 | Hyperspectral image classification method based on correction prototype learning |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114842236A (en) * | 2022-03-22 | 2022-08-02 | 西北工业大学 | Image classification method and device, computer-readable storage medium and electronic device |
| CN114844583A (en) * | 2022-03-30 | 2022-08-02 | 电子科技大学 | End-to-end communication receiving method based on prototype network |
| CN114936615A (en) * | 2022-07-25 | 2022-08-23 | 南京大数据集团有限公司 | Small sample log information anomaly detection method based on characterization consistency correction |
| CN114936615B (en) * | 2022-07-25 | 2022-10-14 | 南京大数据集团有限公司 | Small sample log information anomaly detection method based on characterization consistency correction |
| CN115564960A (en) * | 2022-11-10 | 2023-01-03 | 南京码极客科技有限公司 | Network image label denoising method combining sample selection and label correction |
| CN115564960B (en) * | 2022-11-10 | 2023-03-03 | 南京码极客科技有限公司 | Network image label denoising method combining sample selection and label correction |
| CN115984621A (en) * | 2023-01-09 | 2023-04-18 | 宁波拾烨智能科技有限公司 | Small sample remote sensing image classification method based on restrictive prototype comparison network |
| CN115984621B (en) * | 2023-01-09 | 2023-07-11 | 宁波拾烨智能科技有限公司 | Small sample remote sensing image classification method based on restrictive prototype comparison network |
| CN116778268A (en) * | 2023-04-20 | 2023-09-19 | 江苏济远医疗科技有限公司 | Sample selection deviation relieving method suitable for medical image target classification |
| CN116168257A (en) * | 2023-04-23 | 2023-05-26 | 安徽大学 | Small sample image classification method, device and storage medium based on sample generation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114169442B (en) | 2022-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114169442B (en) | Remote sensing image small sample scene classification method based on double prototype network | |
| CN111476294B (en) | Zero sample image identification method and system based on generation countermeasure network | |
| CN114092832B (en) | High-resolution remote sensing image classification method based on parallel hybrid convolutional network | |
| CN113705597B (en) | Image processing method, device, computer equipment and readable storage medium | |
| CN112819023B (en) | Sample set acquisition method, device, computer equipment and storage medium | |
| CN111898703B (en) | Multi-label video classification method, model training method, device and medium | |
| CN110516095A (en) | Weakly supervised depth Hash social activity image search method and system based on semanteme migration | |
| CN113821670B (en) | Image retrieval method, device, equipment and computer readable storage medium | |
| CN106203483A (en) | A kind of zero sample image sorting technique of multi-modal mapping method of being correlated with based on semanteme | |
| CN111506760B (en) | Depth integration measurement image retrieval method based on difficult perception | |
| CN115311502A (en) | Remote sensing image small sample scene classification method based on multi-scale double-flow architecture | |
| CN113537304A (en) | Cross-modal semantic clustering method based on bidirectional CNN | |
| CN111026887A (en) | Cross-media retrieval method and system | |
| Zhu et al. | Age estimation algorithm of facial images based on multi-label sorting | |
| CN115659254A (en) | Power quality disturbance analysis method for power distribution network with bimodal feature fusion | |
| CN114863175A (en) | Unsupervised multi-source partial domain adaptive image classification method | |
| CN115204171A (en) | Document-level event extraction method and system based on hypergraph neural network | |
| Qin et al. | A wireless sensor network location algorithm based on insufficient fingerprint information | |
| CN112489689B (en) | Cross-database voice emotion recognition method and device based on multi-scale difference countermeasure | |
| CN113837232A (en) | Black box model distillation method based on sample selection and weighting loss function | |
| CN111339258B (en) | University computer basic exercise recommendation method based on knowledge graph | |
| CN112446305A (en) | Pedestrian re-identification method based on classification weight equidistant distribution loss model | |
| CN110288002B (en) | Image classification method based on sparse orthogonal neural network | |
| CN112329830A (en) | Passive positioning track data identification method and system based on convolutional neural network and transfer learning | |
| US20240265207A1 (en) | Training of an object linking model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |