CN116258944B - Remote sensing image classification model sample increment learning method based on double networks - Google Patents
Remote sensing image classification model sample increment learning method based on double networks Download PDFInfo
- Publication number
- CN116258944B CN116258944B CN202310540710.2A CN202310540710A CN116258944B CN 116258944 B CN116258944 B CN 116258944B CN 202310540710 A CN202310540710 A CN 202310540710A CN 116258944 B CN116258944 B CN 116258944B
- Authority
- CN
- China
- Prior art keywords
- model
- remote sensing
- learning
- sensing image
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a remote sensing image classification model sample increment learning method based on a double network, which belongs to the field of data identification and comprises the following steps of using D 0 Training the classification model to obtain a classification model M for identifying N remote sensing image categories 0 The method comprises the steps of carrying out a first treatment on the surface of the Constructing a sample increment learned data stream D 1 ~D t ，D t Class set and D of middle training samples 0 The same; the incremental learning is sequentially carried out, and the process of the t-th incremental learning is as follows: constructing a dual network model including a first network model M t1 And a second network model M t2 ，M t1 With the model M obtained by last learning t‑1 Inter-knowledge distillation for maintaining M t‑1 Old knowledge contained in output, relieving problem of forgetting knowledge, M t2 Will learn D t The resulting output is taken as M t1 And the consistency loss is carried out on the output of the two networks in each training step, so that the model can effectively learn new knowledge, network parameters are updated through a knowledge collaborative algorithm, and the classification performance of the model is improved.
Description
Technical Field
The invention relates to an image classification method, in particular to a remote sensing image classification model sample increment learning method based on a double network.
Background
The classification of the remote sensing image is one of the most basic tasks of the interpretation of the remote sensing image. The traditional remote sensing image classification work is mainly based on manual design features such as scale-invariant feature transformation and visual word bag models. However, because the remote sensing image has a complex geometry and a spatial pattern of differences, the manually designed features may not be able to obtain high-level semantic information of the remote sensing image. In recent years, remote sensing image classification has been greatly advanced under the push of Convolutional Neural Networks (CNNs). Wherein Xue et al extract advanced features of the remote sensing image using a pre-trained CNN as a feature extractor. Xu Suhui and the like extract multi-scale features of the image by using CNN, and realize remote sensing image classification by using a multi-core support vector machine. Song Zhongshan and the like propose to fuse features of different scales by utilizing active rotation polymerization, compensate a convolutional neural network (FAC-CNN) by feature fusion, and apply the convolutional neural network to remote sensing image classification to improve the generalization capability of a model. Qu Zhen and the like propose a new remote sensing image classification model based on an effective channel attention (Efficient Channel Attention, ECA) mechanism, and further improve the classification accuracy and generalization capability of the model.
However, as new remote sensing image data is continuously collected, the remote sensing image classification model based on the deep neural network needs to continuously learn new knowledge in the new data. In order to improve the recognition performance of the remote sensing image, it is important that the model learn new knowledge and keep the memory of old knowledge. Assuming that old data is available, one of the simplest methods is to train on new data and old data with Joint training methods. However, this approach requires the storage of historical data, resulting in wasted storage and computing resources. In practical applications, this is obviously inefficient. If the old data is not stored and used, the new data is directly used for Fine adjustment on the model, and the method is English Fine-tuning and FT is abbreviated. This approach can suffer from forgetfulness, resulting in a decrease in the overall classification performance of the model.
Disclosure of Invention
The invention aims to provide a remote sensing image classification model sample increment learning method based on a double network, which solves the problems, can not forget the capability of the data trained before in the process of continuously learning new data, does not need to store historical data, greatly saves storage space and can improve the overall classification performance of remote sensing images.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: a remote sensing image classification model sample increment learning method based on a double network comprises the following steps of;
(1) Using the original training dataset D 0 Training the classification model to obtain a classification model M capable of identifying N remote sensing image categories 0 The method comprises the steps of carrying out a first treatment on the surface of the The D is 0 The training sample in the training model is a remote sensing image with a remote sensing image type label, D 0 Is c= { C 1 ，C 2 ，…，C N N is the total number of remote sensing image categories;
(2) Constructing a data stream d= { D for sample increment learning 1 ，D 2 ，…，D t T=1, 2, …, training set for the t-th incremental learning is D t The model obtained by learning is M t ，D t Class set and D of middle training samples 0 The same;
(3) The incremental learning is sequentially carried out, and the process of the t-th incremental learning comprises (31) - (33)
(31) Constructing a dual network model including a first network model M t1 And a second network model M t2 Model M obtained by t-1 time increment learning t-1 Initialization results, when t=1, M t-1 Is M 0 ；
(32) By D t Training a first network model M t1 Calculation of M by knowledge distillation t1 And M is as follows t-1 Distillation loss L of (2) kd The method comprises the steps of carrying out a first treatment on the surface of the By D t Training a second network model M t2 Calculate M t2 And M t1 Consistency loss L of (2) con ；
(33) Knowledge collaborative learning;
according to a first network model M t1 Calculates the classification loss L thereof ce ；
Calculating a first network model M t1 Is the complete loss function L of (2) all Updating M through knowledge collaboration policy t2 Weights of (2), and then M t2 Model M obtained as this learning t ；
In the method, in the process of the invention,λandβrespectively is L kd And L ce Is a weight of (2).
As preferable: for training set D t ，Wherein x is i For D t In (i) th training sample, y i Is x i Is a remote sensing image category label, n is D t Is a sample count of the total number of samples in the sample.
As preferable: in step (2), the D 1 The trained classification model is a ResNet network model.
As preferable: in the step (32), the distillation loss L is calculated by the following formula kd ；
Wherein n is D t In the total amount of samples in (a),and->Are respectively M t-1 Neutralization of M t1 A softmax function with a temperature parameter T corresponding to the ith training sample;
wherein the method comprises the steps ofAnd->Respectively D t The ith training sample in (1) at M t-1 Neutralization of M t1 K=1 to n, and represents the kth remote sensing image category.
As preferable: calculating a consistency loss L by con ；
Wherein the method comprises the steps ofAnd->Representing the ith training sample at M t1 And M t2 In (2), MSE () is +.>And->A desired distance therebetween.
As preferable: in step (33), M is calculated by the following formula t1 Classification loss L of (2) ce ；
Wherein n is D t Total number of training samples, x i For D t In (i) th training sample, y i Is x i Is characterized by that the remote sensing image class label,is x i At M t1 Output probability of (a) is determined.
As preferable: updating M in step (33) by knowledge of collaborative policies t Specifically, M is updated by t Weights of (2);
wherein j represents the j-th training in the t-th incremental learning,αis the super-parameter of the smoothing coefficient,and->Respectively denoted as a first network model M t1 Second network model M t2 Is included in the weight information.
Compared with the prior art, the invention has the advantages that: in the prior art, the sample increment is to learn new classification by continuously learning new data sets, so that the recognition performance of the remote sensing image model on old data is reduced due to forgetting knowledge, but the invention is completely different from the prior art, and the invention provides a method for cooperated working of a double-network structure and a model obtained by original sample increment learning on the basis that the sample types in each continuously learned data set are the same, wherein each learning is not to increase the number of the types, but to continuously improve the classification efficiency and the classification precision. The invention can also effectively overcome the problem of forgetting knowledge, and history data is not required to be stored, thereby greatly saving the storage space. Therefore, the incremental learning algorithm for classifying the remote sensing images has wide application prospect.
Wherein, with respect to the first network model: the method aims at preserving old knowledge, if an existing sample increment learning method is adopted, each increment learning is finely tuned on a subsequent training set, the recognition capability of the model on old data is reduced due to forgetting of knowledge, in order to guide a remote sensing image classification model to preserve old knowledge in the process of learning new data, the output probability of each image is expected to be close to the output of an original network for each increment training set, a first network model is introduced, knowledge distillation is adopted, distillation loss between the model and the model obtained by the previous round of learning is calculated, and the method can well encourage the output of the first network model to be close to the output of the previous round of model, so that the method is an improved cross entropy loss and the weight of smaller probability is increased. And in order to avoid the need of large storage requirement, the distillation loss of the invention only depends on the classification model obtained in the previous round, namely, for the t-th incremental learning stage, the distillation loss only depends on M t-1 Is provided.
Regarding the second network model: the purpose of setting is to learn new knowledge, the second network model inputs the data set Dt of the training round, so the second network model has the probability output corresponding to the second network model, but the second network model takes the output of the second network model as the learning target of the first network, and the classification model of the invention can effectively learn new knowledge in new data by continuously maintaining the consistency of the two networks. In the invention, the consistency is realized by calculating the consistency loss of the first network model and the second network model.
Regarding collaborative learning: the invention actually learns the last increment to obtain the classification model M t-1 First network model M constructed by incremental learning t1 Second network model M t2 In combination, jointly control a first network model andthe internal weights of the second network model. In each training step, the first network model trains each data set in a supervised manner by minimizing standard cross entropy loss, namely, in step (33), the classification loss is calculated according to the output of the first network model, then the internal weight of the first network model is updated according to the classification loss, distillation loss and consistency loss, the weight of the second network model is updated by using a knowledge collaborative strategy, the second network model is used as the classification model obtained by the current learning, the second network model updates the internal weight in an updating mode of the knowledge collaborative algorithm, the second network model is not only related to the first network model, the old knowledge and the ability of learning new knowledge can be reserved when the first network model is obtained, and meanwhile, the parameter updating is more robust, and the prediction effect is better.
Drawings
FIG. 1 is a diagram of the overall framework of the present invention;
FIG. 2 is a diagram illustrating a remote sensing image sample of the AID dataset of example 2;
FIG. 3 is the experimental results of example 2 on AID datasets using five methods;
FIG. 4 is the experimental results of example 2 on the NWPU-RESISC45 dataset using five methods;
FIG. 5 is a diagram showing the forgetting of old knowledge using the classification model of the remote sensing image in example 2;
FIG. 6 is M t1 And M t2 And a result graph for classification prediction.
Description of the embodiments
The invention will be further described with reference to the accompanying drawings.
Example 1: referring to fig. 1, a remote sensing image classification model sample increment learning method based on a dual network includes the following steps;
(1) Using the original training dataset D 0 Training the classification model to obtain a classification model M capable of identifying N remote sensing image categories 0 The method comprises the steps of carrying out a first treatment on the surface of the The D is 0 The training sample in the training model is a remote sensing image with a remote sensing image type label, D 0 Is c= { C 1 ，C 2 ，…，C N N is the total number of remote sensing image categories;
(2) Constructing a data stream d= { D for sample increment learning 1 ，D 2 ，…，D t T=1, 2, …, training set for the t-th incremental learning is D t The model obtained by learning is M t ，D t Class set and D of middle training samples 0 The same;
(3) The incremental learning is sequentially carried out, and the process of the t-th incremental learning comprises (31) - (33)
(31) Constructing a dual network model including a first network model M t1 And a second network model M t2 Model M obtained by t-1 time increment learning t-1 Initialization results, when t=1, M t-1 Is M 0 ；
(32) By D t Training a first network model M t1 Calculation of M by knowledge distillation t1 And M is as follows t-1 Distillation loss L of (2) kd The method comprises the steps of carrying out a first treatment on the surface of the By D t Training a second network model M t2 Calculate M t2 And M t1 Consistency loss L of (2) con ；
(33) Knowledge collaborative learning;
according to a first network model M t1 Calculates the classification loss L thereof ce ；
Calculating a first network model M t1 Is the complete loss function L of (2) all Updating M through knowledge collaboration policy t2 Weights of (2), and then M t2 Model M obtained as this learning t ；
In the method, in the process of the invention,λandβrespectively is L kd And L ce Is a weight of (2).
In this embodiment: for training set D t ，Wherein x is i For D t In (i) th training sample, y i Is x i Is a remote sensing image category label, n is D t Is a sample count of the total number of samples in the sample.
In step (2), the D 1 The trained classification model is a ResNet network model.
In the step (32), the distillation loss L is calculated by the following formula kd ；
Wherein n is D t In the total amount of samples in (a),and->Are respectively M t-1 Neutralization of M t1 A softmax function with a temperature parameter T corresponding to the ith training sample;
wherein the method comprises the steps ofAnd->Respectively D t The ith training sample in (1) at M t-1 Neutralization of M t1 K=1 to n, and represents the kth remote sensing image category.
In the present invention, the consistency loss L is calculated by the following formula con ；
Wherein the method comprises the steps ofAnd->Representing the ith training sample at M t1 And M t2 In (2), MSE () is +.>And->A desired distance therebetween.
In step (33), M is calculated by the following formula t1 Classification loss L of (2) ce ；
Wherein n is D t Total number of training samples, x i For D t In (i) th training sample, y i Is x i Is characterized by that the remote sensing image class label,is x i At M t1 Output probability of (a) is determined.
Updating M in step (33) by knowledge of collaborative policies t Specifically, M is updated by t Weights of (2);
wherein j represents the j-th training in the t-th incremental learning,αis the super-parameter of the smoothing coefficient,and->Respectively denoted as a first network model M t1 Second network model M t2 Is included in the weight information.
Example 2: referring to fig. 1 to 4, based on embodiment 1, we perform a remote sensing image sample incremental learning experiment using two challenging remote sensing image data sets AID and NWPU-RESISC 45. Hereinafter, NWPU-RESISC45 is abbreviated NWPU. The AID dataset mainly comprises 30 remote sensing image categories, and totally comprises 10000 marked remote sensing images. The number of images per category is between 220 and 420. The NWPU dataset is 45 classes, each class containing 700 remote sensing images, each image size being 256 x 256 pixels. The two data set images cover a plurality of countries and regions of the world, are photographed under different imaging conditions and seasons, and are suitable for the proposed sample increment learning experimental scene. Fig. 2 is a remote sensing image sample example of an AID dataset.
Construction of data sets for each study: in step (1) of the present invention we need to construct the original training dataset D 0 In step (2) of the present invention, it is also necessary to construct a data stream d= { D for sample increment learning 1 ，D 2 ，…，D t }. For a specific AID dataset and NWPU dataset we set specifically as follows:
for the AID dataset we split it into 5 training sets: training set 1 as D 0 The training set comprises 5000 remote sensing images used for basic model learning, and the rest 4 training sets are used as D in the data stream D 1 To D 4 Each containing 1000 remote sensing images for incremental learning. The size of each picture is cropped to 224 x 224 pixels.
For the NWPU dataset, the same theory is divided into 5 training sets, the 1 st training set is taken as D 0 The method comprises 4500 remote sensing images which are used for basic model learning. The remaining 4 training sets are taken as D in the data stream D 1 To D 4 Each containing 4500 remote sensing images for incremental learning. Incremental learning settings for both data sets are shown in table 1.
Table 1: incremental learning setting table for two data sets
Experiment setting: all experiments were based on Pytorch implementation, using ResNet18 as the trained deep neural network model, for model training, in the raw training dataset D 0 Training was performed on 50 epochs, with an initial learning rate set to 0.01 and a batch size set to 128. In the incremental learning process, D t The upper batch size was set to 64, 20 training rounds were set, the initial learning rate was set to 0.01, and after 15 epochs it was reduced to 0.001. The momentum attenuation coefficient and the weight attenuation coefficient are 0.9 and 0.0001, respectively. In addition, willλAndβthe temperature parameters are respectively set to 0.2 and 1, and the temperature parameter T is set to 2.
Sample increment learning is carried out on two data sets by the method and four methods in the prior art, and the accuracy of the five methods in the increment learning stage is compared.
The four methods in the prior art are respectively as follows:
the method comprises the following steps: the join method, also called a Joint training method, refers to the Joint training of all old data and current new data by a remote sensing image classification model.
The second method is as follows: the FT method is used for Fine tuning new data on a trained remote sensing image classification model, and does not perform any action of retaining old data.
And a third method: lwF, english Learning without Forgetting, the method used for simultaneously optimizing the data contained in the old knowledge and the new knowledge, reduces forgetting by continuously enhancing the stability of the model output on the old knowledge.
The method four: EWC, english Elastic Weight Consolidation, adopts a regularization strategy, and attempts to reduce forgetting by regularizing the weights of the old knowledge network to constrain the changes of important parameters while learning new knowledge.
The results of the experiments based on the AID dataset are shown in table 2 for the examples and the NWPU-RESISC45 dataset are shown in table 3 for the examples.
Table 2: AID dataset experimental result comparison table
Table 3: NWPU-RESISC45 dataset experimental result comparison table
The experimental results on AID and NWPU-RESISC45 datasets using the five methods described above are shown in FIGS. 3 and 4.
As can be seen in combination with tables 2, 3, and fig. 3, 4:
(1) On two remote sensing data sets, the method provided by the invention is superior to the FT method in each incremental learning stage, so that the problem of forgetting knowledge of a remote sensing image classification model is effectively reduced, old knowledge can be well reserved, and the overall classification performance of the model on the remote sensing image can be effectively improved.
(2) For the AID remote sensing dataset, in the second, third and fourth incremental learning processes, the FT method is used, so that the classification performance of the model is not improved, but the accuracy is reduced, and the model is proved to have the effect of forgetting knowledge. In contrast, the methods herein improve classification accuracy during each incremental learning process and outperform other incremental setting methods. After all incremental learning is completed, the proposed method achieves a classification accuracy of 77.40%, which is closest to 79.90% of the join method. The sample increment learning method provided by the invention has the advantages that the classification accuracy of the model to the known remote sensing images can be remarkably improved when new knowledge is learned.
(3) For an NWPU remote sensing dataset, the join method finally achieves 82.17% accuracy after all incremental learning is completed. Compared with other methods, the sample increment learning method provided by the method achieves the optimal result of 80.53 percent, which is closest to the point method. The incremental learning method provided by the method can help the remote sensing image classification model to effectively consolidate old knowledge and learn new knowledge under the condition of not storing old data, and the classification performance of the remote sensing image is continuously improved.
Example 3: referring to fig. 1 to 6, on the basis of embodiment 2, we have performed an ablation experiment to verify the validity of the first network model and the second network model in the collaborative learning knowledge model of the present invention.
Referring to fig. 5, forgetting curves of two remote sensing image classification models for old knowledge are shown, one of which is the FT method and one of which is the ft+kd method, wherein KD represents knowledge distillation, and the method uses knowledge distillation loss while using the FT method. And classifying and identifying the two trained models in all old samples, wherein Total represents the Total number of the old samples in all categories. It can be seen from the figure that the classification accuracy of the model to the old knowledge is remarkably reduced by using the FT method, and the classification performance of the model to the old knowledge can be improved by adopting the FT+KD method under most conditions.
Referring to FIG. 6, a trained network model M is shown t1 And M t2 As a result of classification prediction, the network model M can be seen from the incremental ensemble learning process t2 Ratio model M t1 The recognition effect is better, the classification accuracy is higher, so the model M is used in the experiment t2 As a final model of remote sensing image classification.
See table 4: comparing the overall test accuracy of the method in the last increment learning by adopting the FT method, combining the FT method with the distillation loss, combining the FT method with the consistency loss;
table 4: ablation experiment accuracy contrast table
From the results in table 4, the knowledge distillation and knowledge collaborative learning methods proposed herein can improve the recognition performance of the remote sensing image classification model. Based on knowledge distillation, knowledge collaborative learning is adopted, so that the classification accuracy of the model can be further improved by 6.7 percent compared with a fine adjustment method. This illustrates that they can help the remote sensing image classification model retain old knowledge and learn new knowledge.
In conclusion, the knowledge distillation method is adopted by the first network model and the network model obtained by the previous study to help the model to keep the memory of old knowledge, so that the forgetting of knowledge can be effectively relieved. The second network model takes learning knowledge of the new data as a learning target of the first network. The two networks learn new data together, and training is carried out by adopting a knowledge collaborative algorithm to improve the classification accuracy of the model. The effectiveness of the method is verified on two remote sensing data sets, and experimental results show that the remote sensing image classification model based on sample increment learning provided by the method can effectively reduce forgetting of knowledge and continuously improve the overall recognition performance of the known remote sensing images.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
Claims (5)
1. A remote sensing image classification model sample increment learning method based on a double network is characterized in that: comprises the following steps of;
(1) Using the original training dataset D 0 Training the classification model to obtain a classification model M capable of identifying N remote sensing image categories 0 The method comprises the steps of carrying out a first treatment on the surface of the The D is 0 The training sample in the training model is a remote sensing image with a remote sensing image type label, D 0 Is c= { C 1 ，C 2 ，…，C N N is the total number of remote sensing image categories;
(2) Constructing a data stream d= { D for sample increment learning 1 ，D 2 ，…，D t T=1, 2, …, training set for the t-th incremental learning is D t The model obtained by learning is M t ，D t Class set and D of middle training samples 0 The same;
(3) Sequentially performing incremental learning, wherein the process of the t-th incremental learning comprises (31) - (33);
(31) Constructing a dual network model including a first network model M t1 And a second network model M t2 Model M obtained by t-1 time increment learning t-1 Initial initiationObtained by conversion, when t=1, M t-1 Is M 0 ；
(32) By D t Training a first network model M t1 Calculation of M by knowledge distillation t1 And M is as follows t-1 Distillation loss L of (2) kd The method comprises the steps of carrying out a first treatment on the surface of the By D t Training a second network model M t2 Calculate M t2 And M t1 Consistency loss L of (2) con ；
(33) Knowledge collaborative learning;
according to a first network model M t1 Calculates the classification loss L thereof ce ；
Calculating a first network model M t1 Is the complete loss function L of (2) all Updating M through knowledge collaborative policies t2 Weights of (2), and then M t2 Model M obtained as this learning t ；In the method, in the process of the invention,λandβrespectively is L kd And L ce Weights of (2);
in step (33), M is calculated by the following formula t1 Classification loss L of (2) ce ；
Wherein n is D t Total number of training samples, x i For D t In (i) th training sample, y i Is x i Is a remote sensing image category label->Is x i At M t1 Output probability of (a);
updating M in step (33) by knowledge collaborative policy t Specifically, M is updated by t Weights of (2);
wherein j represents the j-th training in the t-th incremental learning,αis a smoothing coefficient superparameter,/">And->Respectively denoted as a first network model M t1 Second network model M t2 Is included in the weight information.
2. The method for learning the sample increment of the classification model of the remote sensing image based on the double network according to claim 1, wherein the method comprises the following steps of: for training set D t ，In which x is i For D t In (i) th training sample, y i Is x i Is a remote sensing image category label, n is D t Is a sample count of the total number of samples in the sample.
3. The method for learning the sample increment of the classification model of the remote sensing image based on the double network according to claim 1, wherein the method comprises the following steps of: in step (2), D 1 The trained classification model is a ResNet network model.
4. The method for learning the sample increment of the classification model of the remote sensing image based on the double network according to claim 1, wherein the method comprises the following steps of: in the step (32), the distillation loss L is calculated by the following formula kd ；
Wherein n is D t Total amount of samples in>And->Are respectively M t-1 Neutralization of M t1 The ith training sample pair in (a)A softmax function with a temperature parameter T;
wherein->And->Respectively D t The ith training sample in (1) at M t-1 Neutralization of M t1 K=1 to n, and represents the kth remote sensing image category.
5. The method for learning the sample increment of the classification model of the remote sensing image based on the double network according to claim 1, wherein the method comprises the following steps of: calculating a consistency loss L by con ；
Wherein->And->Representing the ith training sample at M t1 And M t2 In (2), MSE () is +.>And->A desired distance therebetween.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310540710.2A CN116258944B (en) | 2023-05-15 | 2023-05-15 | Remote sensing image classification model sample increment learning method based on double networks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310540710.2A CN116258944B (en) | 2023-05-15 | 2023-05-15 | Remote sensing image classification model sample increment learning method based on double networks |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116258944A CN116258944A (en) | 2023-06-13 |
| CN116258944B true CN116258944B (en) | 2023-07-21 |
Family
ID=86681083
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310540710.2A Active CN116258944B (en) | 2023-05-15 | 2023-05-15 | Remote sensing image classification model sample increment learning method based on double networks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116258944B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116977635B (en) * | 2023-07-19 | 2024-04-16 | 中国科学院自动化研究所 | Category increment semantic segmentation learning method and semantic segmentation method |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111325116A (en) * | 2020-02-05 | 2020-06-23 | 武汉大学 | Remote sensing image target detection method capable of evolving based on offline training-online learning depth |
| WO2020197501A1 (en) * | 2019-03-26 | 2020-10-01 | Agency For Science, Technology And Research | Method and system for image classification |
| CN113139487A (en) * | 2021-04-29 | 2021-07-20 | 中国地质大学（武汉） | Building feature learning method oriented to remote sensing image incremental data |
| CN113205142A (en) * | 2021-05-08 | 2021-08-03 | 浙江大学 | Target detection method and device based on incremental learning |
| CN114463574A (en) * | 2021-12-24 | 2022-05-10 | 中国科学院自动化研究所 | Scene classification method and device for remote sensing image |
| CN114529752A (en) * | 2022-01-04 | 2022-05-24 | 成都理工大学 | Sample increment learning method based on deep neural network |
| GB202205999D0 (en) * | 2021-11-09 | 2022-06-08 | Samsung Electronics Co Ltd | Method and apparatus for class incremental learning |
| CN115148327A (en) * | 2022-07-08 | 2022-10-04 | 徐州医科大学 | Multi-party collaborative prototype system and method for intelligent anesthesia decision |
| CN115170872A (en) * | 2022-06-23 | 2022-10-11 | 江苏科技大学 | Class increment learning method based on knowledge distillation |
-
2023
- 2023-05-15 CN CN202310540710.2A patent/CN116258944B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020197501A1 (en) * | 2019-03-26 | 2020-10-01 | Agency For Science, Technology And Research | Method and system for image classification |
| CN111325116A (en) * | 2020-02-05 | 2020-06-23 | 武汉大学 | Remote sensing image target detection method capable of evolving based on offline training-online learning depth |
| CN113139487A (en) * | 2021-04-29 | 2021-07-20 | 中国地质大学（武汉） | Building feature learning method oriented to remote sensing image incremental data |
| CN113205142A (en) * | 2021-05-08 | 2021-08-03 | 浙江大学 | Target detection method and device based on incremental learning |
| GB202205999D0 (en) * | 2021-11-09 | 2022-06-08 | Samsung Electronics Co Ltd | Method and apparatus for class incremental learning |
| CN114463574A (en) * | 2021-12-24 | 2022-05-10 | 中国科学院自动化研究所 | Scene classification method and device for remote sensing image |
| CN114529752A (en) * | 2022-01-04 | 2022-05-24 | 成都理工大学 | Sample increment learning method based on deep neural network |
| CN115170872A (en) * | 2022-06-23 | 2022-10-11 | 江苏科技大学 | Class increment learning method based on knowledge distillation |
| CN115148327A (en) * | 2022-07-08 | 2022-10-04 | 徐州医科大学 | Multi-party collaborative prototype system and method for intelligent anesthesia decision |
Non-Patent Citations (5)
| Title |
|---|
| Incremental Detection of Remote Sensing Objects With Feature Pyramid and Knowledge Distillation;Jingzhou Chen et al.;《 IEEE Transactions on Geoscience and Remote Sensing》;第60卷;1-13 * |
| 基于样本增量学习的遥感影像分类;李雪等;《计算机应用》;1-6 * |
| 基于知识蒸馏的目标检测模型增量深度学习方法;方维维等;《工程科学与技术》;第54卷(第6期);59-66 * |
| 深度学习单帧图像超分辨率重建研究综述;曲延云等;《厦门大学学报(自然科学版》;第60卷(第6期);555-570 * |
| 类别增量学习研究进展和性能评价;朱飞等;《自动化学报》;第49卷(第03期);635-660 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116258944A (en) | 2023-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Blind2unblind: Self-supervised image denoising with visible blind spots | |
| CN110458844B (en) | Semantic segmentation method for low-illumination scene | |
| CN103116766B (en) | A kind of image classification method of encoding based on Increment Artificial Neural Network and subgraph | |
| CN116258944B (en) | Remote sensing image classification model sample increment learning method based on double networks | |
| CN111079655A (en) | Method for recognizing human body behaviors in video based on fusion neural network | |
| Haque et al. | Image denoising and restoration with CNN-LSTM Encoder Decoder with Direct Attention | |
| CN110929610A (en) | Plant disease identification method and system based on CNN model and transfer learning | |
| CN111860587B (en) | Detection method for small targets of pictures | |
| CN112464004A (en) | Multi-view depth generation image clustering method | |
| CN111488917A (en) | Garbage image fine-grained classification method based on incremental learning | |
| CN115424177A (en) | Twin network target tracking method based on incremental learning | |
| Cai et al. | Softer pruning, incremental regularization | |
| CN107341479A (en) | A kind of method for tracking target based on the sparse coordination model of weighting | |
| CN113052254A (en) | Multi-attention ghost residual fusion classification model and classification method thereof | |
| CN111476272B (en) | Dimension reduction method based on structural constraint symmetric low-rank retention projection | |
| CN114818963B (en) | Small sample detection method based on cross-image feature fusion | |
| CN113191445A (en) | Large-scale image retrieval method based on self-supervision countermeasure Hash algorithm | |
| CN116844041A (en) | Cultivated land extraction method based on bidirectional convolution time self-attention mechanism | |
| Zhang et al. | Skip-attention encoder–decoder framework for human motion prediction | |
| CN116523877A (en) | Brain MRI image tumor block segmentation method based on convolutional neural network | |
| CN114882288B (en) | Multi-view image classification method based on hierarchical image enhancement stacking self-encoder | |
| CN113723345B (en) | Domain self-adaptive pedestrian re-identification method based on style conversion and joint learning network | |
| CN114170426A (en) | Algorithm model for classifying rare tumor category small samples based on cost sensitivity | |
| Bu | Deep Clustering Based on Contractive Autoencoder and Self-paced Learning | |
| Tian et al. | Maximum fuzzy entropy and immune clone selection algorithm for image segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |