CN116129219A - SAR target class increment recognition method based on knowledge robust-rebalancing network - Google Patents

Abstract

The invention relates to a SAR target class increment identification method based on a knowledge robust-rebalancing network, which comprises the following steps: acquiring an SAR target image training set; constructing a class increment learning model based on a knowledge robust-rebalancing network, and performing iterative training on the class increment learning model by utilizing an SAR target image training set to obtain a class increment learning model after training; and (3) utilizing the training-completed class increment learning model to realize target identification of the SAR image to be detected. Wherein, class increment learning model includes: an old target teacher sub-network, a new target incremental learning sub-network and a mixed knowledge distillation module. The SAR target class increment recognition method can fully consider and maintain old class feature separability information and enhance the feature separability of new and old classes, and can realize recognition model increment learning to recognize new classes and ensure that the new classes are still robust to old target recognition.

Description

SAR target class increment recognition method based on knowledge robust-rebalancing network

Technical Field

The invention belongs to the technical field of target recognition, and particularly relates to a SAR target class increment recognition method based on a knowledge robust-rebalancing network.

Background

Conventional target recognition methods are generally limited to static environment settings, i.e., it is considered that the tag training data of all target classes to be recognized can be acquired at one time. However, most of the actual recognition environments do not meet the assumption of static conditions, and the marking data of the new target class may be gradually acquired with continuous acquisition of sensor data, so that the actual recognition environments have dynamics.

Currently, radar Automatic Target Recognition (RATR) is widely developed and used in the military and civilian fields. However, RATR is a typical dynamic environment target recognition task, which is directed to a dynamic observation environment in which a target class presents time-varying characteristics, and new targets appear continuously, so that it is difficult to acquire data of all target classes to be recognized simultaneously in a training stage to build a complete target recognition library. In the dynamic environment, once a new target class is observed, the traditional target recognition method needs to retrain the recognition model by combining all new and old target class training data so as to ensure the recognition capability of the recognition model on all observed target classes, and the consumption of time and space resources is remarkable, so that the application of the RATR technology in the real dynamic environment is greatly limited.

The Class Increment Learning (CIL) method is a learning algorithm which can be continuously updated and evolved along with the acquired new class data in a dynamic environment, can update the existing recognition model based on the new target class data, and endows the updated recognition model with the capability of simultaneously recognizing the new and old target classes. The Class Increment Learning (CIL) method can be used for efficiently updating the SAR target recognition model based on the new target class data acquired by the SAR sensor, and meanwhile, the recognition of new and old SAR target classes is realized, so that the recognition model is prevented from being repeatedly trained in a dynamic environment in an inefficient manner.

However, in the mainstream class increment learning method, in the process of updating the recognition model by combining new target class data, a serious catastrophic forgetting problem is faced, that is, the recognition performance of the updated recognition model on the old target class is obviously reduced. In order to solve the above problems, researchers have improved both in terms of how to fully migrate old knowledge using old object recognition models and in terms of how to fully correct unbalanced learning of new and old object class data in model updates.

However, the old knowledge migration and class unbalance correction strategies of the existing class increment learning method only focus on the classification space, neglect key information of the old target class characteristics and influence of class unbalance on the classification learning of the new and old target class characteristics, and obviously reduce the recognition performance of the updated model on the old target class.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a SAR target class increment identification method based on a knowledge robust-rebalancing network. The technical problems to be solved by the invention are realized by the following technical scheme:

the invention provides a SAR target class increment identification method based on a knowledge robust-rebalancing network, which comprises the following steps:

acquiring an SAR target image training set;

constructing a knowledge-based robust-rebalancing network-based class incremental learning model, the class incremental learning model comprising: an old target teacher sub-network, a new target incremental learning sub-network and a mixed knowledge distillation module, wherein,

the old target teacher sub-network is used for providing old target knowledge for the new target incremental learning sub-network and comprises a first trunk feature extraction module and a classification module which are connected in cascade;

the new target incremental learning sub-network is used for incrementally learning new target classes to realize the identification of the new target classes and old target classes at the same time, and comprises a cascaded second trunk feature extraction module and a multi-prototype rebalancing module, wherein the multi-prototype rebalancing module comprises an unbiased classifier learning branch and a separable feature learning branch which are arranged in parallel;

The mixed knowledge distillation module is used for transferring knowledge for correctly identifying an old target from the old target teacher sub-network to the new target incremental learning sub-network;

the loss function of the class increment learning model is as follows:

L _total ＝λ ₁ ·[L _mp +L _frd ]+(1-λ ₁ )·[(1-λ ₂ )·L _ce +λ ₂ ·L _rd ]；

wherein L is _mp Loss function representing a branch of a separable feature learning, L _ce Representing the loss function of the learning branch of the unbiased classifier, L _frd Distillation loss function representing characteristic topological relation, L _rd Representing a classification layer response distillation loss function; lambda (lambda) ₁ Represents a first balance coefficient lambda ₂ Representing a second balance coefficient;

performing iterative training on the class incremental learning model by using the SAR target image training set to obtain a class incremental learning model after training;

and (3) utilizing the training-completed class increment learning model to realize target identification of the SAR image to be detected.

In one embodiment of the invention, acquiring a SAR target image training set comprises:

acquiring a plurality of SAR target images, cutting each SAR target image into images with 64 multiplied by 64 pixels, and giving category labels to each SAR target image;

dividing the plurality of SAR target images into a plurality of groups according to categories, wherein each group comprises 2 category SAR target images;

one group of images is used as an initial training set for training an old target recognition model, the other groups of images are used as a new target class training set for carrying out incremental updating on the class incremental learning model, wherein in each incremental updating, SAR target images of not more than 300 old target classes and the new target class training set are selected to form an incremental learning training set together, and the class incremental learning model is trained and updated.

In one embodiment of the present invention, the first trunk feature extraction module and the second trunk feature extraction module have identical structures, and each include a first convolution layer, a plurality of residual convolution layers and a pooling layer, which are cascaded; wherein, the liquid crystal display device comprises a liquid crystal display device,

the convolution kernel size of the first convolution layer is 3×3, and the number of convolution kernels is set to 64;

the residual convolution layer comprises a second convolution layer, a first Batch Norm layer, a first ReLU nonlinear activation function layer, a third convolution layer, a second Batch Norm layer and a second ReLU nonlinear activation function layer; wherein, the liquid crystal display device comprises a liquid crystal display device,

the input feature map of the residual convolution layer sequentially passes through the second convolution layer, the first Batch Norm layer and the first ReLU nonlinear activation function layer to obtain a first feature map, the first feature map sequentially passes through the third convolution layer and the second Batch Norm layer to be mapped to obtain a second feature map, and the second feature map is added with the input feature map of the residual convolution layer after downsampling to obtain an output feature map of the residual convolution layer through the second ReLU nonlinear activation function layer;

the pooling layer adopts global average pooling.

In one embodiment of the present invention, the classification module includes a first full-connection layer, where the number of input nodes of the first full-connection layer is 512, and the number of output nodes is N, where N represents the total number of classes of the old target class.

In one embodiment of the present invention, the unbiased classifier learning branch includes a second full connection layer, where the number of input nodes in the second full connection layer is 512, the number of output nodes is n+m, and M represents the total number of classes of the newly added target class;

the loss function of the unbiased classifier learning branch is as follows:

/>

wherein B represents the number of training samples input into the learning branch of the unbiased classifier in the current batch training process, y _b Representing the b-th training sample x _b Is a true class label of (c) for a mobile device,

representing training samples x _b Through the second trunk feature extractionThe module and unbiased classifier learn the predicted classification scores from the branches, softmax () represents the normalized exponential function.

In one embodiment of the present invention, the separable feature learning branch includes a third full-connection layer and a fourth full-connection layer in cascade, where the number of input nodes of the third full-connection layer is 512, the number of output nodes of the third full-connection layer is 512, the number of input nodes of the fourth full-connection layer is 512, and the number of output nodes of the fourth full-connection layer is 128;

the loss function of the separable characteristic learning branch is as follows:

wherein B represents the number of training samples for inputting the separable characteristic learning branches in the current batch training process,

representing the d training sample x _d Embedded features obtained through a second trunk feature extraction module and separable feature learning branches, y _d Representing training samples x _d Is a true category label of->

Representing the s-th prototype parameters in the class prototype set of class j,

representing class y _d And the S-th prototype parameter in the corresponding prototype-like set, wherein S represents the total number of prototype parameters in the prototype-like set.

In one embodiment of the invention, the characteristic topological relation distillation loss function is:

wherein, Γ _o Features representing features of samples extracted by old target teacher subnetworksSimilarity matrix Γ _t Feature similarity matrix, W, representing features of samples extracted by new target incremental learning sub-networks _mask Is a mask matrix;

the classifying layer response distillation loss function is:

in the method, in the process of the invention,

pair sample x representing new target incremental learning sub-network output _i Softening output probability at class j, q _j (x _i ) Pair sample x representing old target teacher sub-network output _i Softening output probability at class j.

In one embodiment of the present invention, when the class incremental learning model is iteratively trained by using the SAR target image training set, a class incremental learning model after training is obtained, the method further includes:

And training an old target recognition model by using the initial training set, wherein the trained old target recognition model is used as an old target teacher sub-network of the class incremental learning model.

In one embodiment of the present invention, the training set of SAR target images is used to iteratively train the class incremental learning model to obtain a trained class incremental learning model, which includes the following steps:

step 3a: initializing parameters of the new target increment learning sub-network;

step 3b: 2B SAR target images are selected from the incremental learning training set, wherein the B SAR target images are obtained by randomly sampling the incremental learning training set, and the B SAR target images are obtained by carrying out class equalization sampling on the incremental learning training set;

step 3c: inputting 2B SAR target images into a second trunk feature extraction module of the new target increment learning sub-network to obtain corresponding features, inputting features corresponding to random sampling samples into the separable feature learning branches, calculating loss functions of the corresponding separable feature learning branches, inputting features corresponding to class-balanced sampling samples into the unbiased classifier learning branches, and calculating loss functions of the corresponding unbiased classifier learning branches;

Step 3d: inputting 2B SAR target images into a first trunk feature extraction module of the old target teacher sub-network to obtain corresponding features, calculating corresponding feature topological relation distillation loss functions based on features corresponding to random sampling samples, inputting features corresponding to class-balanced sampling samples into a classification module of the old target teacher sub-network, and calculating corresponding classification layer response distillation loss functions;

step 3e: according to the calculated loss function of the separable characteristic learning branch, the loss function of the unbiased classifier learning branch, the characteristic topological relation distillation loss function and the classification layer response distillation loss function, updating the parameters of the new target increment learning sub-network by using a gradient descent method;

step 3f: repeating the steps 3b-3e for a plurality of times of iterative training until the preset training stopping condition is reached, and finishing one-time increment updating to obtain the training-completed class increment learning model.

In one embodiment of the present invention, the target recognition of the SAR image to be detected is realized by using the trained class increment learning model, which comprises the following steps:

inputting the SAR image to be detected into a new target increment learning sub-network of a class increment learning model with training completed, after extracting the characteristics of the SAR image to be detected, inputting the extracted characteristics into a learning branch of the unbiased classifier to obtain the prediction classification probability of the SAR image to be detected, and obtaining the prediction class of the SAR image to be detected according to the maximum probability.

Compared with the prior art, the invention has the beneficial effects that:

according to the SAR target class increment recognition method based on the knowledge robust-rebalancing network, target recognition of SAR images to be detected is achieved by utilizing a class increment learning model based on the knowledge robust-rebalancing network which is completed through training, the class increment learning model enhances the feature separability between new and old various samples based on multi-prototype measurement learning at a feature level through a class imbalance correction strategy combining designed feature separability learning and unbiased classifier learning, class weight balanced classifiers are learned based on class balance sampling optimization at a classification level, and the problem that old target class test samples are easily misjudged as new target classes due to new and old class imbalance learning of the network is fully corrected; through the designed old target knowledge robust retention strategy combining feature topological relation retention and classification prediction retention, old class feature separability learned by an old target recognition model is retained in incremental update based on topological relation distillation at a feature level, and old class correct prediction capability learned by the old target recognition model is retained based on response knowledge distillation at a classification level. The SAR target class increment recognition method can fully consider and maintain old class feature separability information and enhance the feature separability of new and old classes, and can realize recognition model increment learning to recognize new classes and ensure that the new classes are still robust to old target recognition.

The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention, as well as the preferred embodiments thereof, together with the following detailed description of the invention, given by way of illustration only, together with the accompanying drawings.

Drawings

FIG. 1 is a flowchart of a SAR target class increment recognition method based on a knowledge robust-rebalancing network provided by an embodiment of the present invention;

FIG. 2 is a schematic diagram of a framework of a knowledge-based robust-rebalancing network-based class incremental learning model according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a residual convolution layer according to an embodiment of the present disclosure;

fig. 4 is a graph of the recognition accuracy of the method of the present invention and the existing class increment method based on the MSTAR dataset increment experiment according to the embodiment of the present invention.

Detailed Description

In order to further explain the technical means and effects adopted by the invention to achieve the preset aim, the following describes in detail a SAR target class increment recognition method based on a knowledge robust-rebalance network according to the invention with reference to the attached drawings and the detailed description.

The foregoing and other features, aspects, and advantages of the present invention will become more apparent from the following detailed description of the preferred embodiments when taken in conjunction with the accompanying drawings. The technical means and effects adopted by the present invention to achieve the intended purpose can be more deeply and specifically understood through the description of the specific embodiments, however, the attached drawings are provided for reference and description only, and are not intended to limit the technical scheme of the present invention.

Example 1

Referring to fig. 1, fig. 1 is a flowchart of a method for identifying SAR target class increments based on a knowledge robust-rebalancing network according to an embodiment of the present invention, as shown in the fig. 1, the method for identifying SAR target class increments based on the knowledge robust-rebalancing network according to the embodiment includes:

step 1: acquiring an SAR target image training set;

in an alternative embodiment, step 1 comprises:

step 1a: acquiring a plurality of SAR target images, cutting each SAR target image into images with 64 multiplied by 64 pixels, and giving category labels to each SAR target image;

in this embodiment, all 5172 images in the SAR vehicle target data set MSTAR (moving and stationary target acquisition) are adopted as a sample set, each of 2746 images with a radar working pitch angle of 17 ° in the sample set is cut into images with pixels of 64×64, and each image is given a category label of 1-10, that is, 2746 images are divided into 10 categories.

Step 1b: dividing the multiple SAR target images into multiple groups according to categories, wherein each group comprises 2 category SAR target images;

in this embodiment, 10 classes of SAR target images are randomized into 5 groups, each group containing 2 classes of SAR target images.

Step 1c: one group of images is used as an initial training set for training an old target recognition model, the other groups of images are used as a new target class training set for carrying out incremental updating on a class incremental learning model, wherein in each incremental updating, SAR target images of not more than 300 old target classes and the new target class training set are selected to form an incremental learning training set together, and the class incremental learning model is trained and updated.

In this embodiment, the first group of images is used as an initial training set, the other 4 groups are used as new target class training sets, and in each incremental update, 300 SAR target images of the old target class and the new target class training sets are selected to form an incremental learning training set together, and it is to be noted that the proportion of each class in the SAR target images of the 300 old target classes is equal.

Step 2: constructing a knowledge-based robust-rebalance network-based class incremental learning model;

Please refer to fig. 2 in combination with a schematic diagram of a framework of a knowledge-based robust-rebalance network-based class incremental learning model according to an embodiment of the present invention, which includes an old target teacher sub-network M _o New target incremental learning sub-network M _t And a mixed knowledge distillation module HKD.

Wherein, old target teacher sub-network M _o For learning the subnetwork M for new target increments _t Providing old target knowledge, in this embodiment, old target teacher sub-network M _o Is a given old object identification network.

Optionally, old target teacher subnetwork M _o The method comprises a first trunk feature extraction module F and a classification module C which are connected in cascade.

In an alternative embodiment, the first trunk feature extraction module F includes a cascaded first convolution layer, a plurality of residual convolution layers, and a pooling layer; the classification module C comprises a first full connection layer.

In this embodiment, the first trunk feature extraction module F includes nine residual convolution layers, and the specific structure of the first trunk feature extraction module F is: first layer convolution layer- & gt first layer residual convolution layer- & gt second layer residual convolution layer- & gt third layer residual convolution layer- & gt fourth layer residual convolution layer- & gt fifth layer residual convolution layer- & gt sixth layer residual convolution layer- & gt seventh layer residual convolution layer- & gt eighth layer residual convolution layer- & gt ninth layer residual convolution layer- & gt pooling layer. Wherein the convolution kernel size of the first layer of convolution layers is 3×3, and the number of convolution kernels is set to 64.

Referring to fig. 3, a schematic structural diagram of a residual convolution layer according to an embodiment of the present invention is shown, where the residual convolution layer in this embodiment includes a second convolution layer, a first Batch Norm layer, a first ReLU nonlinear active function layer, a third convolution layer, a second Batch Norm layer, and a second ReLU nonlinear active function layer. The input feature map of the residual convolution layer sequentially passes through a second convolution layer, a first Batch Norm layer and a first ReLU nonlinear activation function layer to obtain a first feature map, the first feature map sequentially passes through a third convolution layer and a second Batch Norm layer to be mapped to obtain a second feature map, and the second feature map is added with the input feature map of the residual convolution layer subjected to downsampling and then passes through the second ReLU nonlinear activation function layer to obtain an output feature map of the residual convolution layer. In this embodiment, the convolution kernel sizes of the first to ninth residual convolution layers are all 3×3, and the number of convolution kernels is set to 64, 64, 128, 128, 256, 256, 256, 512, 512 in order;

in this embodiment, the pooling layer is set to global average pooling, and the feature map of 512×h×w output by the ninth layer residual convolution layer is mapped to a feature vector of 512×1. The number of input nodes of the first full connection layer is 512, the number of output nodes is N, and N represents the total category number of the old target category.

Wherein, new target increment learns the sub-network M _t Is in the old target teacher sub-network M _o The obtained network is used for incrementally learning the new target class to realize the identification of the new target class and the old target class at the same time.

In an alternative embodiment, the new target incremental learning sub-network M _t Comprising a cascade of a secondA main feature extraction module F' and a multi-prototype re-balance module MPR. In this embodiment, the second trunk feature extraction module F' and the first trunk feature extraction module F have the same structure, and are not described herein. The multi-prototype re-balance module MPR comprises an unbiased classifier learning branch and a separable feature learning branch arranged in parallel.

Optionally, the unbiased classifier learning branch includes a second full connection layer, where the number of input nodes in the second full connection layer is 512, the number of output nodes is n+m, and M represents the total class number of the newly added target class. The loss function of the unbiased classifier learning branch is as follows:

wherein B represents the number of training samples input into the learning branch of the unbiased classifier in the current batch training process, y _b Representing the b-th training sample x _b Is a true class label of (c) for a mobile device,

representing training samples x _b The prediction classification score obtained by the second trunk feature extraction module and the unbiased classifier learning branch is softmax () to represent the normalized exponential function. / >

In this embodiment, the unbiased classifier learns the loss function L of the branch _ce And the cross entropy loss of the prediction label and the real label obtained by the branch of the B samples of the current batch through the unbiased classifier learning is represented. It should be noted that, the above B samples come from class-balanced sampling of the incremental learning training set, so as to ensure that the classifier learns unbiased class weight parameters.

Optionally, the separable feature learning branch includes a third full-connection layer and a fourth full-connection layer, where the number of input nodes of the third full-connection layer is 512, the number of output nodes is 512, the number of input nodes of the fourth full-connection layer is 512, and the number of output nodes is 128. The loss function of the separable characteristic learning branch is as follows:

wherein B represents the number of training samples for inputting the separable characteristic learning branches in the current batch training process,

representing the d training sample x _d Embedded features obtained through a second trunk feature extraction module and separable feature learning branches, y _d Representing training samples x _d Is a true category label of->

Representing the s-th prototype parameters in the class prototype set of class j,

representing class y _d And the S-th prototype parameter in the corresponding prototype-like set, wherein S represents the total number of prototype parameters in the prototype-like set.

In the present embodiment, the loss function L of the branching of the separability feature learning branch _mp Representing embedded features of B samples of a current batch obtained by separable feature learning branches

And various prototype collections->

Loss of multi-prototype comparison between. Optimizing the loss, can restrict the embedded characteristics of various samples ∈ ->

Approach to a prototype set of this class in feature space

Far from other prototype sets->

And further, the method ensures that all new target class features and old target class features extracted by the second trunk feature extraction module F' have good intra-class compactness and inter-class separability. It should be noted that, the above B samples are derived from random sampling of the incremental learning training set, so as to ensure that the second trunk feature extraction module F' has a good separable feature extraction capability under the original data distribution.

Wherein the mixed knowledge distillation module HKD is intended to address the old target teacher sub-network M by constraining it _o And new target incremental learning subnetwork M _t Consistent with the classification decision response for the purpose of correctly identifying knowledge of the old target from the old target teacher sub-network M _o Migration to new target incremental learning subnetwork M _t . In this embodiment, the hybrid knowledge distillation module HKD includes a characteristic topological relationship distillation loss function L _frd And a classification layer response distillation loss function L _rd Two-part distillation loss function, i.e. L _HKD ＝L _frd +L _rd 。

Wherein, the characteristic topological relation distillation loss function is:

wherein, Γ _o Feature similarity matrix Γ representing features of samples extracted by old target teacher sub-network _t Feature similarity matrix, W, representing features of samples extracted by new target incremental learning sub-networks _mask Is a mask matrix.

In particular, the method comprises the steps of,

wherein z is _i And->

Respectively represent sample x _i Inputting old target teacher sub-network M _o First trunk feature extraction module F and new target incremental learning sub-network M _t Second stem feature extraction die of (a)The features obtained by block F', representing cosine similarity calculations, W _mask For extracting Γ _o And Γ _t Related to the feature similarity term of the old object class sample.

In the present embodiment, the characteristic topological relation distills the loss function L _frd To restrict the input of B samples of the current batch into the old target teacher sub-network M _o And new target incremental learning subnetwork M _t Is consistent with the similarity between the features of (a). This loss will be the old target teacher subnetwork M _o The learned discrimination knowledge with high similarity of old object class characteristics and low similarity between classes is transferred to the new object incremental learning sub-network M _t So that the new target increment learns the sub-network M _t The extraction capability of the old target separable features is reserved.

Wherein, the response distillation loss function of the classifying layer is:

in the method, in the process of the invention,

pair sample x representing new target incremental learning sub-network output _i Softening output probability at class j, q _j (x _i ) Pair sample x representing old target teacher sub-network output _i Softening output probability at class j.

In particular, the method comprises the steps of,

representing sample x _i Learning sub-network M at new target increment _t Is the classification score, τ is the softening temperature coefficient,

representing sample x _i In old target teacher sub-network M _o Is a classification score of (c).

In the present embodiment, the classification layer responds to the distillation loss function L _rd To restrict the input of B samples of the current batch into the old target teacher sub-network M _o And new target incremental learning subnetwork M _t Is consistent with the classification response of the new target incremental learning sub-network M _t The correct predictive power for the old object class is maintained.

Step 3: iterative training is carried out on the class increment learning model by utilizing the SAR target image training set, and a class increment learning model after training is completed is obtained;

in this embodiment, when performing iterative training on the class incremental learning model by using the SAR target image training set, the training-completed class incremental learning model is obtained, the method further includes: and training the old target recognition model by using the initial training set, wherein the trained old target recognition model is used as an old target teacher sub-network of the class increment learning model.

In this embodiment, the specific training process for training the old target recognition model is consistent with the conventional target recognition training method under the static environment setting, and will not be described herein.

In an alternative embodiment, the iterative training of the class incremental learning model using the SAR target image training set includes the steps of:

step 3a: initializing parameters of a new target increment learning sub-network;

optionally, the parameters of the new target incremental learning sub-network are randomly initialized based on normal distribution, and the set iterative training parameters of the class incremental learning model include the iterative times Q and the maximum iterative times Q, wherein Q is greater than or equal to 100 and q=0, and in the embodiment, q=200.

Step 3b: 2B SAR target images are selected from the incremental learning training set, wherein the B SAR target images are obtained by randomly sampling the incremental learning training set, and the B SAR target images are obtained by carrying out class equalization sampling on the incremental learning training set;

optionally, the B SAR target images obtained by random sampling are denoted as D _rad B SAR target images obtained by class equalization sampling are marked as D _bal 。

Wherein the sampling probability of selecting one sample from the j-th training data is that

Wherein q is E [0,1 ]]，n _j The number of SAR target images included in the j-th training data is represented, and when q=0 corresponds to random sampling, q=1 corresponds to class equalization sampling, in this embodiment, b=128.

Step 3c: inputting 2B SAR target images into a second trunk feature extraction module of a new target increment learning sub-network to obtain corresponding features, inputting features corresponding to random sampling samples into separable feature learning branches, calculating loss functions of the corresponding separable feature learning branches, inputting features corresponding to class-balanced sampling samples into unbiased classifier learning branches, and calculating loss functions of the corresponding unbiased classifier learning branches;

in the present embodiment, D is _rad And D _bal Inputting new target increment learning sub-network M _t The second trunk feature extraction module F' of the model (C) is subjected to global average pooling to obtain corresponding K-dimensional features

And +.>

Features corresponding to randomly sampled samples->

Inputting a separable characteristic learning branch of the multi-prototype re-balance module MPR to obtain a mapped embedded characteristic +.>

And based on->

Calculating a loss function L of a separable feature learning branch _mp . The corresponding feature of the class equalization sample is +.>

Input polytypeUnbiased classifier learning branch of rebalancing module MPR, obtaining mapped classification score +.>

And based on->

Calculating a loss function L of an unbiased classifier learning branch _ce 。

Step 3d: inputting 2B SAR target images into a first trunk feature extraction module of an old target teacher sub-network to obtain corresponding features, calculating corresponding feature topological relation distillation loss functions based on features corresponding to random sampling samples, inputting features corresponding to class equilibrium sampling samples into a classification module of the old target teacher sub-network, and calculating corresponding classification layer response distillation loss functions according to prediction classification scores of the old target teacher sub-network and prediction classification scores of new target increment learning sub-networks;

In the present embodiment, D is _rad And D _bal Inputting old target teacher sub-network M _o The first trunk feature extraction module F of the model (C) is subjected to global average pooling to obtain corresponding K-dimensional features

And +.>

Separately calculating old target teacher sub-network M _o Extracted features->

Feature similarity matrix->

And new target incremental learning subnetwork M _t Extracted features->

Feature similarity matrix of (a)

And is based on gamma _o And F-shaped structure _t Calculating characteristic topological relation distillation loss function L _frd . The corresponding feature of the class equalization sample is +.>

Inputting the obtained classification scores into a classification module C to obtain mapped classification scores

And based on old target teacher sub-network M _o Class equalization sample predicted classification score O _bal And new target incremental learning subnetwork M _t Class-balanced sample predicted class score>

Calculating a loss classification layer response distillation loss function L _rd 。

Step 3e: according to the calculated loss function of the separable characteristic learning branch, the unbiased classifier learns the loss function of the branch, the characteristic topological relation distills the loss function and the classifying layer responds to the distillation loss function, and the gradient descent method is utilized to update the parameters of the new target increment learning sub-network;

in the present embodiment, the gradient descent method is used to calculate the loss function L based on the step 3d _mp And L _ce Loss function L calculated in step 3e _frd And L _rd Learning the sub-network M for new target increment according to the loss function of the class increment learning model _t And updating parameters of each layer to obtain an updated class increment identification model.

The loss function of the class increment learning model is as follows:

L _total ＝λ ₁ ·[L _mp +L _frd ]+(1-λ ₁ )·[(1-λ ₂ )·L _ce +λ ₂ ·L _rd ]；

wherein lambda is ₁ Represents a first balance coefficient lambda ₂ Representing a second balance systemA number;

alternatively, the training process uses a momentum SGD optimizer, with a weight regularization term of 1×10 ^-4 The momentum factor is 0.9, the initial learning rate is set to 0.5, and the initial learning rate is attenuated to 1/10 of the original learning rate in 120 iterations and 160 iterations respectively.

Step 3f: repeating the step bc-3e for multiple iterative training until the preset training stopping condition is reached, and finishing one-time increment updating to obtain the training-completed class increment learning model.

In this embodiment, the preset training stop condition is that the iteration number Q reaches the maximum iteration number Q, or that the loss function of the class-increment learning model reaches a preset threshold.

After the incremental update is completed, if a new incremental update is to be performed, the class incremental learning model training process is performed according to the class incremental learning model training process in the steps 3b-3f, and then the class incremental learning model training is performed by using the new incremental learning training set.

It is worth to say that, the class increment learning model based on the knowledge robust-rebalancing network constructed in this embodiment performs update training of the new target increment recognition model based on a small amount of stored old target class samples and acquired new target class samples, and in the update training process, key feature knowledge and key classification knowledge for correctly recognizing the old target in the old target recognition model are migrated to the new target increment recognition model, so as to ensure the robustness of the learned old target knowledge; the balance learning of the new target increment recognition model on the new target class sample and the old target class sample is realized based on the rebalancing module, the problems that the feature separability of the old target class extracted by the new target increment recognition model is poor, and the preference exists in classification prediction are avoided, and the problem that the recognition performance of the old target class cannot be effectively maintained while the new target class is incrementally learned in the prior art is solved.

Step 4: and (3) utilizing the training-completed class increment learning model to realize target identification of the SAR image to be detected.

In an alternative embodiment, the SAR image to be detected is input into a new target increment learning sub-network of the trained class increment learning model, the second trunk feature extraction module extracts features of the SAR image to be detected, the extracted features are input into an unbiased classifier learning branch to obtain the prediction classification probability of the SAR image to be detected, and the prediction class of the SAR image to be detected is obtained according to the maximum probability.

In this embodiment, each image in 2426 images with radar working pitch angle of 15 ° in a sample set formed by target data of SAR vehicles is cut into images with pixels of 64×64, class labels of each image belong to 1-10, and all cut images are divided into test sets corresponding to 5 sets of training sets according to class labels, so as to be used as SAR images to be tested, and the training-completed class incremental learning model is tested.

The SAR target class increment recognition method based on the knowledge robust-rebalancing network can fully consider the feature separability information of the old class and enhance the feature separability of the new class, and can realize recognition model increment learning to recognize the new class and ensure that the new class is still robust to old target recognition.

Referring to the graph of the recognition accuracy based on the MSTAR data set increment experiment provided by the embodiment of the invention and the conventional class increment method, as shown in FIG. 4, the recognition accuracy of the method of the invention is better than that of the conventional class increment method.

It is to be understood that the above description is only one specific example of the present invention and is not to be construed as limiting the invention in any way, and that various modifications and changes in form and detail may be made to the invention without departing from the principles and construction thereof, which, however, are intended to be within the scope of the appended claims. For example, a possible alternative is to optimize the hidden feature extraction of new and old object class samples based on example contrast learning at the feature level, describe the learned feature topology relationship of the old object recognition model based on any metric mode, but still aim to ensure the robustness of the old object feature relationship and the separability of the new and old object features, and the improvement of the basic optimization strategy is still within the scope of the claims of the invention without departing from the concept of the invention.

It should be noted that in this document relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that an article or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in an article or apparatus that comprises the element. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

The foregoing is a further detailed description of the invention in connection with the preferred embodiments, and it is not intended that the invention be limited to the specific embodiments described. It will be apparent to those skilled in the art that several simple deductions or substitutions may be made without departing from the spirit of the invention, and these should be considered to be within the scope of the invention.

Claims (10)

1. A knowledge-based robust-rebalancing network-based SAR target class increment identification method, comprising:

acquiring an SAR target image training set;

constructing a knowledge-based robust-rebalancing network-based class incremental learning model, the class incremental learning model comprising: an old target teacher sub-network, a new target incremental learning sub-network and a mixed knowledge distillation module, wherein,

the old target teacher sub-network is used for providing old target knowledge for the new target incremental learning sub-network and comprises a first trunk feature extraction module and a classification module which are connected in cascade;

the new target incremental learning sub-network is used for incrementally learning new target classes to realize the identification of the new target classes and old target classes at the same time, and comprises a cascaded second trunk feature extraction module and a multi-prototype rebalancing module, wherein the multi-prototype rebalancing module comprises an unbiased classifier learning branch and a separable feature learning branch which are arranged in parallel;

the mixed knowledge distillation module is used for transferring knowledge for correctly identifying an old target from the old target teacher sub-network to the new target incremental learning sub-network;

the loss function of the class increment learning model is as follows:

L _total ＝λ ₁ ·[L _mp +L _frd ]+(1-λ ₁ )·[(1-λ ₂ )·L _ce +λ ₂ ·L _rd ]；

Wherein L is _mp Loss function representing a branch of a separable feature learning, L _ce Representing the loss function of the learning branch of the unbiased classifier, L _frd Distillation loss function representing characteristic topological relation, L _rd Representing a classification layer response distillation loss function; lambda (lambda) ₁ Represents a first balance coefficient lambda ₂ Representing a second balance coefficient;

performing iterative training on the class incremental learning model by using the SAR target image training set to obtain a class incremental learning model after training;

and (3) utilizing the training-completed class increment learning model to realize target identification of the SAR image to be detected.

2. The knowledge-based robust-rebalancing network based SAR target class delta identification method according to claim 1, wherein obtaining the SAR target image training set comprises:

acquiring a plurality of SAR target images, cutting each SAR target image into images with 64 multiplied by 64 pixels, and giving category labels to each SAR target image;

dividing the plurality of SAR target images into a plurality of groups according to categories, wherein each group comprises 2 category SAR target images;

one group of images is used as an initial training set for training an old target recognition model, the other groups of images are used as a new target class training set for carrying out incremental updating on the class incremental learning model, wherein in each incremental updating, SAR target images of not more than 300 old target classes and the new target class training set are selected to form an incremental learning training set together, and the class incremental learning model is trained and updated.

3. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 1, wherein the first trunk feature extraction module and the second trunk feature extraction module are identical in structure and each comprise a first convolution layer, a plurality of residual convolution layers and a pooling layer which are cascaded; wherein, the liquid crystal display device comprises a liquid crystal display device,

the convolution kernel size of the first convolution layer is 3×3, and the number of convolution kernels is set to 64;

the residual convolution layer comprises a second convolution layer, a first Batch Norm layer, a first ReLU nonlinear activation function layer, a third convolution layer, a second Batch Norm layer and a second ReLU nonlinear activation function layer; wherein, the liquid crystal display device comprises a liquid crystal display device,

the input feature map of the residual convolution layer sequentially passes through the second convolution layer, the first Batch Norm layer and the first ReLU nonlinear activation function layer to obtain a first feature map, the first feature map sequentially passes through the third convolution layer and the second Batch Norm layer to be mapped to obtain a second feature map, and the second feature map is added with the input feature map of the residual convolution layer after downsampling to obtain an output feature map of the residual convolution layer through the second ReLU nonlinear activation function layer;

the pooling layer adopts global average pooling.

4. The method for incremental identification of SAR target classes based on a knowledge robust-rebalance network according to claim 1, wherein said classification module comprises a first fully connected layer having an input node number of 512 and an output node number of N, wherein N represents the total class number of the old target class.

5. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 4, wherein the unbiased classifier learning branch includes a second full-connection layer, the number of input nodes of the second full-connection layer is 512, the number of output nodes is N+M, and M represents the total class number of the newly added target class;

the loss function of the unbiased classifier learning branch is as follows:

wherein B represents the number of training samples input into the learning branch of the unbiased classifier in the current batch training process, y _b Representing the b-th training sample x _b Is a true class label of (c) for a mobile device,

representing training samples x _b The prediction classification score obtained by the second trunk feature extraction module and the unbiased classifier learning branch is softmax () to represent the normalized exponential function.

6. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 5, wherein the separable feature learning branch comprises a third fully connected layer and a fourth fully connected layer in cascade, the number of input nodes of the third fully connected layer is 512, the number of output nodes of the third fully connected layer is 512, the number of input nodes of the fourth fully connected layer is 512, and the number of output nodes of the fourth fully connected layer is 128;

The loss function of the separable characteristic learning branch is as follows:

wherein B represents the number of training samples for inputting the separable characteristic learning branches in the current batch training process,

representing the d training sample x _d Embedded features obtained through a second trunk feature extraction module and separable feature learning branches, y _d Representing training samples x _d Is a true category label of->

Representing the s-th prototype parameter in the class prototype set of class j +.>

Representing class y _d And the S-th prototype parameter in the corresponding prototype-like set, wherein S represents the total number of prototype parameters in the prototype-like set.

7. The knowledge-based robust-rebalancing network based SAR target class delta identification method of claim 6, wherein said characteristic topological relation distillation loss function is:

wherein, Γ _o Feature similarity matrix Γ representing features of samples extracted by old target teacher sub-network _t Feature similarity matrix, W, representing features of samples extracted by new target incremental learning sub-networks _mask Is a mask matrix;

the classifying layer response distillation loss function is:

/>

in the method, in the process of the invention,

representing new target incrementsLearning sub-network output pair sample x _i Softening output probability at class j, q _j (x _i ) Pair sample x representing old target teacher sub-network output _i Softening output probability at class j.

8. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 2, wherein, before performing iterative training on the class increment learning model by using the SAR target image training set to obtain a trained class increment learning model, further comprising:

and training an old target recognition model by using the initial training set, wherein the trained old target recognition model is used as an old target teacher sub-network of the class incremental learning model.

9. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 8, wherein the iterative training of the class increment learning model is performed by using the SAR target image training set to obtain a trained class increment learning model, comprising the steps of:

step 3a: initializing parameters of the new target increment learning sub-network;

step 3b: 2B SAR target images are selected from the incremental learning training set, wherein the B SAR target images are obtained by randomly sampling the incremental learning training set, and the B SAR target images are obtained by carrying out class equalization sampling on the incremental learning training set;

Step 3c: inputting 2B SAR target images into a second trunk feature extraction module of the new target increment learning sub-network to obtain corresponding features, inputting features corresponding to random sampling samples into the separable feature learning branches, calculating loss functions of the corresponding separable feature learning branches, inputting features corresponding to class-balanced sampling samples into the unbiased classifier learning branches, and calculating loss functions of the corresponding unbiased classifier learning branches;

step 3d: inputting 2B SAR target images into a first trunk feature extraction module of the old target teacher sub-network to obtain corresponding features, calculating corresponding feature topological relation distillation loss functions based on features corresponding to random sampling samples, inputting features corresponding to class-balanced sampling samples into a classification module of the old target teacher sub-network, and calculating corresponding classification layer response distillation loss functions;

step 3e: according to the calculated loss function of the separable characteristic learning branch, the loss function of the unbiased classifier learning branch, the characteristic topological relation distillation loss function and the classification layer response distillation loss function, updating the parameters of the new target increment learning sub-network by using a gradient descent method;

Step 3f: repeating the steps 3b-3e for a plurality of times of iterative training until the preset training stopping condition is reached, and finishing one-time increment updating to obtain the training-completed class increment learning model.

10. The knowledge-based robust-rebalancing network-based SAR target class increment recognition method according to claim 1, wherein the training-completed class increment learning model is utilized to realize target recognition of the SAR image to be detected, comprising:

inputting the SAR image to be detected into a new target increment learning sub-network of a class increment learning model with training completed, after extracting the characteristics of the SAR image to be detected, inputting the extracted characteristics into a learning branch of the unbiased classifier to obtain the prediction classification probability of the SAR image to be detected, and obtaining the prediction class of the SAR image to be detected according to the maximum probability.

Images