CN115082790A - Remote sensing image scene classification method based on continuous learning - Google Patents
Remote sensing image scene classification method based on continuous learning Download PDFInfo
- Publication number
- CN115082790A CN115082790A CN202210712526.7A CN202210712526A CN115082790A CN 115082790 A CN115082790 A CN 115082790A CN 202210712526 A CN202210712526 A CN 202210712526A CN 115082790 A CN115082790 A CN 115082790A
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- sensing image
- feature
- feature subspace
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Abstract
The invention discloses a remote sensing image scene classification method based on continuous learning, which comprises the steps of obtaining a remote sensing image data set to be classified and a remote sensing image training data set at the current moment; the image category in the remote sensing image data set to be classified is larger than the image category in the remote sensing image training data set; training a remote sensing image classification model at the previous moment by adopting a remote sensing image training data set; classifying the remote sensing image data set to be classified through the trained remote sensing image classification model; according to the method, the remote sensing image classification model at the last moment is updated by using the remote sensing image training data set, and the second loss function is added into the loss function in the updating process, so that the remote sensing image classification model can keep the classification precision of the original remote sensing image, and the generalization capability of the remote sensing image classification model is further improved.
Description
Technical Field
The invention belongs to the technical field of remote sensing image classification, and particularly relates to a remote sensing image scene classification method based on continuous learning.
Background
With the development of earth observation technology, computer technology, artificial intelligence technology and the like, the era of remote sensing big data has come. Especially, the innovation of communication technologies such as 5G and the like gradually makes intelligent interconnection of everything possible. Under the condition, the application technology of remote sensing edge intelligence is developed, the remote sensing data processing and analyzing technology is applied to the scenes of satellite on-orbit processing interpretation, unmanned aerial vehicle dynamic real-time tracking, large-scale urban environment reconstruction, unmanned identification planning and the like, and a large amount of transmission bandwidth, processing time and resource consumption can be undoubtedly saved.
Deep learning has been extensively studied in the field of intelligent interpretation of remote sensing data by virtue of its powerful feature self-learning and application generalization capabilities, and numerous intelligent interpretation models of remote sensing data based on deep neural network models are proposed in succession and popularized.
However, in the analysis of the edge remote sensing data, a new task and a new category are layered endlessly, and the existing method is poor in generalization capability for continuously input data and has a catastrophic forgetting problem, that is, a model can only be applied to a current task and a remote sensing image sample, and cannot adapt or expand with the lapse of time.
Disclosure of Invention
The invention aims to provide a remote sensing image scene classification method based on continuous learning, and the method is used for solving the problem of poor model generalization capability in the continuous learning process.
The invention adopts the following technical scheme: a remote sensing image scene classification method based on continuous learning comprises the following steps:
acquiring a remote sensing image data set to be classified and a remote sensing image training data set at the current moment; the image category in the remote sensing image data set to be classified is larger than the image category in the remote sensing image training data set;
training a remote sensing image classification model at the previous moment by adopting a remote sensing image training data set;
and classifying the remote sensing image data set to be classified through the trained remote sensing image classification model.
Further, training the remote sensing image classification model at the previous moment by adopting the remote sensing image training data set comprises:
by using
As a loss function of the remote sensing image classification model; wherein the content of the first and second substances,
is a loss function of the remote sensing image classification model,
a first loss function calculated from the predicted class and the true class of each remote sensing image in the remote sensing image training dataset,
and calculating a second loss function based on the feature subspace of the remote sensing image training data set and the feature subspace of the previous moment, wherein the feature subspace is used for representing a feature matrix of the remote sensing image training data set.
Further, the second loss function calculation method is as follows:
and calculating the distance between the feature subspace of the current moment and the feature subspace of the previous moment, and taking the distance as a second loss function.
Further, calculating the distance between the feature subspace at the current time and the feature subspace at the previous time comprises:
calculating the standard orthogonal basis of the feature subspace of the current moment and the standard orthogonal basis of the feature subspace of the previous moment;
and calculating the sum of sine values of an included angle between the two standard orthogonal bases, and taking the sum as the distance between the feature subspace of the current moment and the feature subspace of the previous moment.
Further, calculating the orthonormal basis of the feature subspace at the current time comprises:
extracting a characteristic diagram of each remote sensing image in the remote sensing image training data set through the remote sensing image classification model at the previous moment;
combining the characteristic graphs to obtain a characteristic matrix;
and carrying out singular value decomposition on the characteristic matrix to obtain a standard orthogonal basis of the characteristic subspace at the current moment.
Further, after performing singular value decomposition on the feature matrix, the method further includes:
and taking the left singular matrix of the feature matrix as a standard orthogonal basis of the feature subspace of the current moment.
Further, the second loss function is specifically:
wherein the content of the first and second substances,
representing a feature subspace S at the current time (t) And the feature subspace S of the last time (t-1) Is the feature subspace S (t) Of the orthonormal basis and the feature subspace S (t-1) The angle vector between the orthonormal bases of (a).
Further, the included angle vector includes a plurality of included angles, and the calculation method of the plurality of included angles is as follows:
wherein the content of the first and second substances,
is the first angle in the vector of angles,
as a feature subspace S (t-1) The first of the orthonormal bases of (a),
as a feature subspace S (t) The first of the orthonormal bases of (a),
is the second angle in the vector of angles,
as a feature subspace S (t-1) The second of the orthonormal bases of (a),
as a feature subspace S (t) The second one of the orthonormal bases of (a),
is the (b) th included angle in the included angle vector, b is the number of included angles in the included angle vector,
as a feature subspace S (t-1) The (b) th orthonormal basis of (a),
as a feature subspace S (t) The b-th orthonormal basis of (1).
Further, the orthonormal basis is obtained by performing singular value decomposition on the feature matrix.
The other technical scheme of the invention is as follows: a remote sensing image scene classification device based on continuous learning comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor executes the computer program to realize the remote sensing image scene classification method based on continuous learning.
The invention has the beneficial effects that: according to the method, the remote sensing image classification model at the last moment is updated by using the remote sensing image training data set, and the second loss function is added into the loss function in the updating process, so that the remote sensing image classification model can keep the classification precision of the original remote sensing image, and the generalization capability of the remote sensing image classification model is further improved.
Drawings
FIG. 1 is a schematic diagram of a network architecture of a remote sensing image classification model in an embodiment of the present invention;
FIG. 2 is a flowchart of a method for classifying scenes of remote sensing images based on continuous learning according to an embodiment of the present invention;
FIG. 3 is a training schematic diagram of a remote sensing image classification model at time t in the embodiment of the invention;
FIG. 4 is a schematic diagram showing comparison of accuracy rates of continuous classification of remote sensing images of various methods in the verification embodiment of the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
In the edge remote sensing data analysis, when new data is available and the model needs to be updated, the model cannot keep the performance requirement on the original task. For example, in a task of detecting and identifying targets such as airplanes and ships based on satellite images, a currently common method is to train models individually for different targets, even for subdivided target models, and this way is complex to implement and cumbersome to apply. The satellite images can be updated in a large amount every day, and the existing static model cannot be updated in time and newly added data can not be utilized. Therefore, how to perform continuous learning to maintain high accuracy and high performance of the model algorithm becomes a challenge to be solved.
The difficulty in solving the problem of continuous learning is to learn a new task while forgetting the old task as little as possible. The prior art mainly comprises a regularization-based method, a data reproduction-based method, a model structure expansion-based method and the like.
Regularization-based methods: an additional regularization term is introduced into the loss function, and the previous knowledge is consolidated by limiting the change degree of the model parameters while learning a new task. The storage of original input data, privacy protection and memory requirement reduction are avoided.
When continuous tasks are faced, the regularization-based method reduces forgetting through the change degree of constraint model parameters, but the higher level of data is not constrained, so that the characteristics of the data of the previous task and the characteristics of the next task have larger difference in the aspects of scale, semantics and the like, and the effect of solving the problem of catastrophic forgetting is poor because the new characteristics tend to cover the old characteristics in order to realize the new task.
Adding a memory storage unit: a storage module is added to the model to store the previous data and periodically play back the intersection data of the previously learned knowledge and the new sample. By storing samples of previous tasks or applying a generating model to generate pseudo samples, the stored samples are played back while a new task is learned, and the problem of catastrophic forgetting is alleviated by balancing training data of different tasks.
One common drawback of the continuous learning method with the addition of memory storage modules is that they require explicit storage of old task information, which results in a large working memory requirement, and therefore additional storage space is added, and when the storage space is limited (such as remote sensing satellites), the model has a poor effect of solving the forgetting problem.
The method based on parameter isolation comprises the following steps: the method assigns different model parameters to each task to prevent any possible forgetting. When the architecture size is not limited, new branches can be grown for new tasks while freezing previous task parameters or dedicating model copies to each task. Alternatively, the architecture remains static, with a fixed portion allocated for each task. Previous task parts are masked off during the training of the new task or applied at the parameter level or unit level. The model is divided into subsets which are specially used for each task, so that the task adaptability of the model is improved, and catastrophic forgetting is avoided under the condition of not depending on historical data.
With the increase of network branches along with the increase of the number of tasks, the neural network architecture is increased continuously, and the scalability of the model is reduced directly. Since the number of tasks and the sample size cannot be known in advance in the continuous learning scenario, it is inevitable to define enough storage resources in advance without making a strong assumption about the probability distribution of the input training samples, increasing the storage cost. Moreover, the available load of the edge remote sensing equipment is limited, hardware equipment such as storage, processing and the like has great limitation, and a large amount of remote sensing historical data cannot be stored.
Therefore, the invention firstly defines the concept of the feature subspace, explicitly models the description mode of the features, restricts the feature subspace shared by different tasks through the self-knowledge distillation way, reduces the feature gap between different tasks, relieves the catastrophic forgetting, and improves the classification precision of continuous learning without increasing the additional storage cost. In the invention, the feature subspace can be regarded as the knowledge about the current remote sensing image learned by the network (namely, the remote sensing image classification model), and the orthonormal basis can be regarded as a description operator of the knowledge about the current remote sensing image learned by the network.
The invention discloses a remote sensing image scene classification method based on continuous learning, which comprises the following steps: acquiring a remote sensing image data set to be classified and a remote sensing image training data set at the current moment; the image category in the remote sensing image data set to be classified is larger than the image category in the remote sensing image training data set; training a remote sensing image classification model at the previous moment by adopting a remote sensing image training data set; and classifying the remote sensing image data set to be classified through the trained remote sensing image classification model.
According to the method, the remote sensing image classification model at the last moment is updated by using the remote sensing image training data set, and the second loss function is added into the loss function in the updating process, so that the remote sensing image classification model can keep the classification precision of the original remote sensing image, and the generalization capability of the remote sensing image classification model is further improved.
The invention combines the thought of feature subspace and knowledge distillation, can help the model to refine and store more perfect knowledge about data on the premise of not increasing the storage cost, and helps the model to reduce catastrophic forgetting. The network model is shown in fig. 1, and the model is composed of a feature extraction part, a subspace construction part and a classification part.
The process flow is shown in figure 2. Firstly, obtaining remote sensing image classification data D containing new classes at time t (t) Each sample in the data
All have labels
The entire sample set is denoted X (t) The entire set of tags is denoted as Y (t) Then there is D (t) ＝(X (t) ，Y (t) )。
In this embodiment, the training phase of the invention is taken as an example. Classifying data D using newly acquired remote sensing image (t) Remote sensing image training data set Train for constructing model (t) And Test data Test (t) (ii) a Secondly, training sample X at time t (t) Model M input to previous round (t-1) Obtaining and recording the feature subspace S of the training sample of the previous round in the previous round model (t-1) The feature attributes are the feature attributes of the feature map of the training sample data of the current round; thereafter, the data D is classified by using the remote sensing image (t) And a feature subspace S (t-1) For model M (t-1) Training to obtain updated model M (t) (ii) a Finally, saving the updated model of the training parameters of the current round as M (t) And the updated model M is used before the next round of training (t) And carrying out continuous classification tasks on the remote sensing image data set to be classified. The following describes the data set construction and model training in detail.
In the embodiment of the invention, training data Train of the model in the t-th round of training (t) . When the t-th round of training begins, classifying the acquired remote sensing image data D containing the new category (t) 75% of the total image data are subjected to data amplification, i.e. random cutting, flipping, contrast and grey-scale adjusted D (t)* Training the model as training data, and removing the current training data after each round of training is finished, namely the training data of each round only comprises the currently acquired remote sensing image training data set, Train (t) ＝D (t)* 。
In the training phase of the model, the Test data of the model in the t-th round of Test is Test (t) . At the end of the t round of training, the whole sample is taken
The model was evaluated as test data at 25%.
And (4) training the model of the t-th round, as shown in FIG. 3. Firstly, training a t-th round model by a sample X (t) Inputting the model M of the previous round (t-1) Performing the following steps; second, training data training sample X is extracted using a feature extractor section (t) Each sample of
Characteristic f of i (t) (ii) a After that, feature f i (t) Obtaining a feature subspace S by a subspace construction part (t-1) (ii) a At the same time, feature f i (t) The prediction results are generated by the classification section.
In the embodiment of the invention, the training of the remote sensing image classification model at the previous moment by adopting the remote sensing image training data set comprises the following steps: by using
As a loss function of the remote sensing image classification model; wherein the content of the first and second substances,
and classifying the loss function of the model for the remote sensing image, namely a first loss function.
Calculating a cross entropy loss function according to the prediction class and the real class of each remote sensing image in the remote sensing image training data set, and obtaining training data Train (t) A first entropy loss function calculated from the first prediction class probability and the true label for each sample.
And calculating a second loss function based on the feature subspace of the remote sensing image training data set and the feature subspace of the previous moment, wherein the feature subspace is used for representing a feature matrix of the remote sensing image training data set. After the loss function is determined, the model is trained by optimizing the loss function by using a gradient descent algorithm to obtain a model M with updated parameters (t) 。
In the embodiment of the present invention, the second loss function calculation method includes: and calculating the distance between the feature subspace of the current moment and the feature subspace of the previous moment, and taking the distance as a second loss function.
Specifically, calculating the distance between the feature subspace at the current time and the feature subspace at the previous time includes: calculating the standard orthogonal basis of the feature subspace of the current moment and the standard orthogonal basis of the feature subspace of the previous moment; and calculating the sum of sine values of an included angle between the two standard orthogonal bases, and taking the sum as the distance between the feature subspace of the current moment and the feature subspace of the previous moment.
More specifically, calculating the orthonormal basis of the feature subspace at the current time includes: extracting a characteristic diagram of each remote sensing image in the remote sensing image training data set through the remote sensing image classification model at the previous moment; combining the characteristic graphs to obtain a characteristic matrix; and carrying out singular value decomposition on the characteristic matrix to obtain a standard orthogonal basis of the characteristic subspace at the current moment.
Preferably, after the singular value decomposition of the feature matrix, the method further includes: and taking the left singular matrix of the feature matrix as a standard orthogonal basis of the feature subspace of the current moment.
Specifically, to measure the Distance (FSD) between Feature subspaces, it is defined as the similarity of the orthonormal bases of two Feature subspaces, i.e. the sum of the sine values of the included angles between the orthonormal bases of two Feature subspaces. The second loss function is a self-knowledge distillation loss function constructed by minimizing the distance between two feature subspaces, specifically:
wherein the content of the first and second substances,
representing a feature subspace S at the current time (t) And the feature subspace S of the last time (t-1) Is the feature subspace S (t) Of the orthonormal basis and the feature subspace S (t-1) The angle vector between the orthonormal bases of (a),
||·|| 1 is a norm of 1.
Preferably, the included angle vector includes a plurality of included angles, and the calculation method of the plurality of included angles is as follows:
wherein the content of the first and second substances,
is corresponding to model M (t-1) The orthogonal basis of (a) is,
is corresponding to the model M (t) The orthogonal basis of (2).
Is the first angle in the vector of angles,
as a feature subspace S (t-1) The first of the orthonormal bases of (a),
as a feature subspace S (t) The first of the orthonormal bases of (a),
is the second angle in the vector of angles,
as a feature subspace S (t-1) The second of the orthonormal bases of (a),
as a feature subspace S (t) The second of the orthonormal bases of (a),
is the (b) th included angle in the included angle vector, b is the number of included angles in the included angle vector,
as a feature subspace S (t-1) The (b) th orthonormal basis of (a),
as a feature subspace S (t) The b-th orthonormal basis of (1).
In the present embodiment, the orthonormal basis is obtained by subjecting the feature matrix to singular value decomposition. First, each sample x in the training data is extracted using a feature extractor section i Characteristic f of i After a batch of samples pass through a feature extractor, a feature matrix F ═ F is formed 1 ...f b ]And b is the number of samples in the batch. Next, Singular Value Decomposition (SVD) is performed on the obtained feature matrix F:
F＝U∑V，
wherein, U left singular matrix, V right singular matrix, and Sigma diagonal matrix; in this embodiment, the matrix U is used as an orthogonal basis in the model feature subspace S, and then the subspace is characterized using the orthonormal basis of the feature subspace.
That is, the training pattern at time tThis X (t) Model M input to previous round (t-1) Obtaining and recording the feature subspace S of the training sample of the previous round in the previous round model (t-1) Of the orthonormal base U (t-1) The feature subspace updated at the training time is S (t) 。
In one embodiment, the model employs a Resnet34 network. The subspace construction part uses SVD decomposition without parameter updating. The model is optimized by using an Adam optimizer, and the parameters are set as defaults.
In addition, in order to prove the feasibility of the method of the invention, the following verification examples are also specifically carried out:
an experimental data set is first determined. The verified continuous Learning bench for Remote Sensing (CLRS) data set consisted of 15000 Remote Sensing images and was divided into 25 scene categories, namely airports, bare land, beaches, bridges, businesses, deserts, farmlands, forests, golf courses, highways, industries, lawns, mountains, overpasses, parks, parking lots, playgrounds, ports, railways, train stations, houses, rivers, runways, stadiums and storage tanks. Each class has 600 images with a size of 256 x 256. The resolution of the image ranges between 0.26m and 8.85 m. The 15000 pictures are from over 100 countries and regions around the world.
The experimental setup was performed next. Dividing 25 scene classes into 5 mutually disjoint parts, wherein each part comprises 5 classes of images, each part is set as a task, 75% is used as training, and 25% is used as testing, namely the training data set by each task comprises 5 scene classes and 2250 images, and the testing data also comprises 5 scene classes and 750 images.
However, the test data after each round of training includes the test data set for each task, i.e., t × 5 scene classes, t × 750 images, and t is the number of tasks. In this task setup, the test set contains the scene classes that appeared in the training set in the previous task. The classifier needs to not only learn the scene class currently seen but also distinguish all scene classes seen so far. The model must therefore be able to learn new scene classes quickly and not forget previously learned class knowledge. Models are required to be able to accurately predict new scene classes without losing the accuracy of prediction of already learned classes.
In order to verify the high efficiency of the method, the invention is compared with the current mainstream continuous learning method, including EWC and LWF. The result shows that on the test set, the average classification accuracy of the method at each stage on the continuous task can be at least 4% higher than that of all other methods, as shown in fig. 4 and table 1, which fully illustrates the high efficiency of the method in the classification of the edge intelligent remote sensing scene.
TABLE 1 method comparison of continuous classification results of remote sensing images
| Methods | Task1 | Task2 | Task3 | Task4 | Task5 |
| LwF | 98.59％ | 50.59％ | 34.00％ | 23.52％ | 14.32％ |
| EWC | 98.06％ | 47.33％ | 31.25％ | 22.56％ | 12.78％ |
| Fine-tuning | 98.89％ | 41.51％ | 32.93％ | 21.41％ | 10.88％ |
| Ours | 98.71％ | 67.89％ | 41.85％ | 27.93％ | 22.94％ |
In summary, the present invention first proposes the concept of feature subspace in the continuous learning task. And explicitly performing singular value decomposition on the feature matrix of the data extracted by the model to obtain a left singular matrix of the feature matrix, namely a standard orthogonal base of the feature subspace, representing the feature subspace by using the standard orthogonal base, and simultaneously measuring the similarity of the feature subspace by using the standard orthogonal base to further constrain the feature subspace. Furthermore, the method is simple and easy to operate. The invention proposes to limit the consistency of different task feature subspaces in a self-knowledge distillation way. The characteristic subspace of different task data on the same model at different moments is limited through a knowledge distillation loss function, so that the self-knowledge distillation of the model is realized, and meanwhile, the catastrophic forgetting is reduced.
That is to say, the invention uses the feature subspace to depict different data features of tasks at different stages, explicitly uses the feature of data extracted by the subspace modeling network to unify the feature description mode, and reduces the semantic drift problem caused by data difference. It is proposed to limit the consistency of different task feature subspaces in a self-knowledge distillation manner. By reserving the feature subspace instead of directly reserving the features or samples, the requirement for storage resources is reduced, meanwhile, the model is enabled to keep the stability of the feature space all the time in the optimization process, the catastrophic forgetting phenomenon is relieved, and the continuous classification precision of the edge remote sensing images is improved on the premise of not additionally increasing the storage resources.
The invention also discloses a remote sensing image scene classification device based on continuous learning, which comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor realizes the remote sensing image scene classification method based on continuous learning when executing the computer program.
The device can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The apparatus may include, but is not limited to, a processor, a memory. Those skilled in the art will appreciate that the apparatus may include more or fewer components, or some components in combination, or different components, and may also include, for example, input-output devices, network access devices, etc.
The Processor may be a Central Processing Unit (CPU), and the Processor may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage may in some embodiments be an internal storage unit of the device, such as a hard disk or a memory of the device. The memory may also be an external storage device of the apparatus in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. provided on the apparatus. Further, the memory may also include both an internal storage unit and an external storage device of the apparatus. The memory is used for storing an operating system, application programs, a BootLoader (BootLoader), data, and other programs, such as program codes of the computer programs. The memory may also be used to temporarily store data that has been output or is to be output.
It should be noted that, for the specific content of the above-mentioned apparatus, since the same concept is based on, the specific functions and the technical effects brought by the method embodiment of the present invention, reference may be made to the method embodiment section specifically, and details are not described here.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment. Those of ordinary skill in the art will appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Claims (10)
1. A remote sensing image scene classification method based on continuous learning is characterized by comprising the following steps:
acquiring a remote sensing image data set to be classified and a remote sensing image training data set at the current moment; the image category in the remote sensing image data set to be classified is larger than the image category in the remote sensing image training data set;
training a remote sensing image classification model at the previous moment by adopting the remote sensing image training data set;
and classifying the remote sensing image data set to be classified through the trained remote sensing image classification model.
2. The method for remote sensing image scene classification based on continuous learning of claim 1, wherein the training of the remote sensing image classification model at the previous moment by using the remote sensing image training data set comprises:
by using
As a loss function of the remote sensing image classification model; wherein the content of the first and second substances,
a loss function for the remote sensing image classification model,
for a first loss function calculated from the predicted class and the actual class of each remote sensing image in the training dataset of remote sensing images,
and calculating a second loss function based on a feature subspace of the remote sensing image training data set and a feature subspace of the previous moment, wherein the feature subspace is used for representing a feature matrix of the remote sensing image training data set.
3. The remote sensing image scene classification method based on continuous learning of claim 2, characterized in that the second loss function calculation method is as follows:
and calculating the distance between the feature subspace of the current moment and the feature subspace of the previous moment, and taking the distance as the second loss function.
4. The remote sensing image scene classification method based on continuous learning of claim 3, wherein the calculation of the distance between the feature subspace of the current time and the feature subspace of the previous time comprises:
calculating the standard orthogonal basis of the feature subspace of the current moment and the standard orthogonal basis of the feature subspace of the previous moment;
and calculating the sum of sine values of an included angle between the two standard orthogonal bases, and taking the sum as the distance between the feature subspace of the current moment and the feature subspace of the previous moment.
5. The method for classifying remote sensing image scenes based on continuous learning as claimed in claim 4, wherein calculating the orthonormal basis of the feature subspace at the current moment comprises:
extracting a characteristic diagram of each remote sensing image in the remote sensing image training data set through a remote sensing image classification model at the previous moment;
combining the characteristic graphs to obtain a characteristic matrix;
and carrying out singular value decomposition on the characteristic matrix to obtain a standard orthogonal basis of the characteristic subspace of the current moment.
6. The method for classifying scenes of remote sensing images based on continuous learning of claim 5, wherein after the singular value decomposition of the feature matrix, the method further comprises:
and taking the left singular matrix of the feature matrix as a standard orthogonal basis of the feature subspace of the current moment.
7. The method for classifying remote sensing image scenes based on continuous learning according to claim 3 or 4, wherein said second loss function is specifically:
wherein the content of the first and second substances,
representing a feature subspace S at the current time (t) And the feature subspace S of the last time (t-1) Is the feature subspace S (t) Is standard ofCross base and feature subspace S (t-1) The angle vector between the orthonormal bases of (a).
8. The method for classifying scenes of remote sensing images based on continuous learning of claim 7, wherein the vector of included angles comprises a plurality of included angles, and the calculation method of the included angles comprises the following steps:
wherein the content of the first and second substances,
is the first angle in the vector of angles,
as a feature subspace S (t-1) The first of the orthonormal bases of (a),
as a feature subspace S (t) The first of the orthonormal bases of (a),
is the second angle in the vector of angles,
as a feature subspace S (t-1) The second of the orthonormal bases of (a),
as a feature subspace S (t) The second of the orthonormal bases of (a),
is the (b) th included angle in the included angle vector, b is the number of included angles in the included angle vector,
as a feature subspace S (t-1) The (b) th orthonormal basis of (a),
as a feature subspace S (t) The b-th orthonormal basis of (1).
9. The method for classifying scenes in remote sensing images based on continuous learning of claim 8, wherein the orthonormal basis is obtained by performing singular value decomposition on the feature matrix.
10. A remote sensing image scene classification device based on continuous learning, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to realize a remote sensing image scene classification method based on continuous learning according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210712526.7A CN115082790A (en) | 2022-06-22 | 2022-06-22 | Remote sensing image scene classification method based on continuous learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210712526.7A CN115082790A (en) | 2022-06-22 | 2022-06-22 | Remote sensing image scene classification method based on continuous learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115082790A true CN115082790A (en) | 2022-09-20 |
Family
ID=83253956
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210712526.7A Pending CN115082790A (en) | 2022-06-22 | 2022-06-22 | Remote sensing image scene classification method based on continuous learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115082790A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115410051A (en) * | 2022-11-02 | 2022-11-29 | 华中科技大学 | Continuous image classification method and system based on re-plasticity inspiration |
| CN115661708A (en) * | 2022-10-24 | 2023-01-31 | 南京理工大学 | Edge video analysis method based on active continuous learning |
| CN116168311A (en) * | 2023-04-18 | 2023-05-26 | 中国人民解放军战略支援部队航天工程大学 | Unmanned aerial vehicle remote sensing monitoring system and method for forest diseases and insect pests |
-
2022
- 2022-06-22 CN CN202210712526.7A patent/CN115082790A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115661708A (en) * | 2022-10-24 | 2023-01-31 | 南京理工大学 | Edge video analysis method based on active continuous learning |
| CN115661708B (en) * | 2022-10-24 | 2023-08-25 | 南京理工大学 | Edge Video Analysis Method Based on Active Continuous Learning |
| CN115410051A (en) * | 2022-11-02 | 2022-11-29 | 华中科技大学 | Continuous image classification method and system based on re-plasticity inspiration |
| CN115410051B (en) * | 2022-11-02 | 2023-01-24 | 华中科技大学 | Continuous image classification method and system based on re-plasticity inspiration |
| CN116168311A (en) * | 2023-04-18 | 2023-05-26 | 中国人民解放军战略支援部队航天工程大学 | Unmanned aerial vehicle remote sensing monitoring system and method for forest diseases and insect pests |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10636169B2 (en) | Synthesizing training data for broad area geospatial object detection | |
| Grilli et al. | A review of point clouds segmentation and classification algorithms | |
| CN115082790A (en) | Remote sensing image scene classification method based on continuous learning | |
| Saltori et al. | Cosmix: Compositional semantic mix for domain adaptation in 3d lidar segmentation | |
| Hu et al. | A comparison and strategy of semantic segmentation on remote sensing images | |
| Lu et al. | P_SegNet and NP_SegNet: New neural network architectures for cloud recognition of remote sensing images | |
| Zhuang et al. | Normal endmember spectral unmixing method for hyperspectral imagery | |
| Iyer et al. | Trinity: A no-code AI platform for complex spatial datasets | |
| Yang et al. | Toward country scale building detection with convolutional neural network using aerial images | |
| Li et al. | An aerial image segmentation approach based on enhanced multi-scale convolutional neural network | |
| Neris et al. | FPGA-Based implementation of a CNN architecture for the on-Board processing of very high-resolution remote sensing images | |
| Djenouri et al. | Deep learning based decomposition for visual navigation in industrial platforms | |
| Quesada-Barriuso et al. | GPU accelerated waterpixel algorithm for superpixel segmentation of hyperspectral images | |
| Bhil et al. | Recent progress in object detection in satellite imagery: A review | |
| Wang et al. | Cross-domain learning using optimized pseudo labels: toward adaptive car detection in different weather conditions and urban cities | |
| Wu et al. | Instant water body variation detection via analysis on remote sensing imagery | |
| Fadavi Amiri et al. | Improving image segmentation using artificial neural networks and evolutionary algorithms | |
| CN116664826A (en) | Small sample point cloud semantic segmentation method | |
| Alemohammad | The case for open-access ML-ready geospatial training data | |
| Anilkumar et al. | An enhanced multi-objective-derived adaptive deeplabv3 using g-rda for semantic segmentation of aerial images | |
| Wu | Big data classification of remote sensing image based on cloud computing and convolutional neural network | |
| Datcu et al. | Deep learning training and benchmarks for earth observation images: Data sets, features, and procedures | |
| Karpagam et al. | Leveraging CNN Deep Learning Model for Smart Parking | |
| Chattopadhyay et al. | A test case for application of convolutional neural networks to spatio-temporal climate data: Re-identifying clustered weather patterns | |
| Stromann et al. | Learning to integrate vision data into road network data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |