CN109508655B - SAR target recognition method based on incomplete training set of twin network - Google Patents

Abstract

The invention discloses a SAR target recognition method based on an incomplete training set of a twin network, which uses k-NN algorithm in traditional machine learning as a reference, extracts n samples from each category of the training set as representative of samples of the category to form a support set, for example, m categories are shared in a classification task, and the total number of the samples in the support set is m x n. When classifying, the samples to be classified and the supporting samples in the supporting set are input into the network together, each sample in the supporting set and the samples to be classified form an input pair, the input pair is respectively input into two inputs of the twin network to extract the characteristics, the characteristics extracted by the two samples are subjected to difference, the difference of the characteristics is judged to obtain the similarity degree of the samples to be classified and the samples in a certain class in the supporting set, and finally the samples to be classified are classified into the class of the samples with the highest similarity degree in the sample and the supporting set.

Description

SAR target recognition method based on incomplete training set of twin network

Technical Field

The invention relates to an SAR target recognition method based on an incomplete training set of a twin network, and belongs to the field of computer vision.

Background

Synthetic Aperture Radar (SAR) is a method for acquiring ground data with high resolution and high penetrability all day long and all weather, and has high civil and commercial values. Interpreting SAR images can obtain a lot of useful information, and therefore interpretation of SAR images is an important part in SAR practical applications. Traditional machine learning and deep learning are two main methods of SAR image interpretation. SAR imaging is more stable than other sensors and is not susceptible to weather, light and other conditions. Another advantage of SAR is that a large amount of ground information data can be generated. It is difficult to manually process such a large amount of data.

The computer vision image processing technology based on traditional machine learning and deep learning can well solve the problem of large data volume. The traditional machine learning method has a strict mathematical theory as a support, the requirement on computational resources is lower than that of a neural network, and meanwhile, the classification and identification precision can also meet the requirement to a certain extent. With the improvement of the computing power of the computer, the related processing methods based on the neural network are greatly different in color, and the classification and identification precision of the methods is far higher than that of the machine learning method. However, the classification and recognition methods of the neural network depend on a large amount of training data, so that the large amount of training data cannot be obtained in practical application and real conditions, which requires a large amount of labor cost for collection and labeling. Too few training samples eventually lead to the occurrence of neural network overfitting, i.e., with high classification or recognition accuracy on the training samples, but with very poor results in testing and practical use.

In addition, the neural network has the defect of poor model interpretability, so that the direction of guidance is difficult to find in the process of optimization. The twin network effectively combines the advantages of traditional machine learning and deep learning, utilizes the neural network to replace the artificially designed feature extractor and then combines the traditional machine learning decision strategy for classification. The combination not only plays the coding capability of the neural network for fully utilizing the computing resources, but also partially avoids the problem that the modeling result of the conventional neural network is difficult to interpret, so that the subsequent improvement and optimization are more trace and circulated. Under realistic conditions, not only is a great deal of effort required to label the sample, but it is also faced with the situation that some of the class samples may be missing. The number of samples is increased by the twin network aiming at the characteristic training mode phase change of small samples, so that the classification precision is improved, and the overfitting condition is weakened. Meanwhile, the classification strategy of distinguishing sample categories by utilizing the distance between sample features also enables the optimization thought to be clearer.

Disclosure of Invention

The invention mainly aims to provide an SAR target recognition method based on an incomplete training set of a twin network.

The invention provides small sample recognition aiming at real data of SAR under real conditions after fully researching the related direction of small sample target recognition. The prediction type of the output sample is different from that of the output sample after the sample to be classified is directly input when the conventional deep learning method is used for classifying. The invention uses k-NearestNeighbor (k-NN) algorithm in traditional machine learning as reference, and extracts n samples from each category of the training set as representative of the samples of the category to form a supporting set, for example, m categories are shared in a classification task, and the total number of the samples in the supporting set is m. When classifying, the samples to be classified and the supporting samples in the supporting set are input into the network together, each sample in the supporting set and the samples to be classified form an input pair, the input pair is respectively input into two inputs of the twin network to extract the characteristics, the characteristics extracted by the two samples are subjected to difference, the difference of the characteristics is judged to obtain the similarity degree of the samples to be classified and the samples in a certain class in the supporting set, and finally the samples to be classified are classified into the class of the samples with the highest similarity degree in the sample and the supporting set.

The technical scheme of the invention specifically mainly comprises the following technical contents:

1. the convolutional neural network extracts SAR target features. The characteristics of the SAR target are extracted by using a plurality of layers of convolution neural networks with different convolution kernels. The weighting of the convolution kernel is used for obtaining advanced features and the pooling layer is used for carrying out the dimension reduction and enhancing the robustness of the network, and meanwhile, the ReLU function is used as an activation function to introduce nonlinear factors so that the neural network can solve nonlinear classification tasks.

2. k-nearest neighbor algorithm. The principle of the k-nearest neighbor algorithm is adopted in classification, the k-nearest neighbor algorithm is simple and effective in the problem of small samples, resources of a training set can be fully utilized, and the robustness of a model can be enhanced by a proper k value.

3. Data enhancement. In order to avoid the over fitting of the model caused by the small sample, the single sample is not directly input for classification during training, but is combined with other samples in the training set to form input pairs, and the combination mode greatly improves the training data and effectively avoids the over fitting, as shown in fig. 2.

4. Back propagation algorithm (BP algorithm). In the invention, the multi-layer neural network updates the weights and the offsets of the convolution kernel and the full connection layer by using a BP algorithm. The basis of the BP algorithm is that the gradient descent method consists of two parts, namely excitation propagation and weight updating. The combined image pair is input into a network for forward propagation to obtain a prediction result, and the prediction result is compared with a label to obtain an error. The output errors are then back propagated to obtain the errors for the nodes of each hidden layer. And updating the convolution kernel, the full-connection layer weight W and the bias b by using a chain rule and a gradient descent method.

The SAR target recognition method based on the incomplete training set of the twin network comprises the following implementation flow:

step 1, specification of data sets: unified size, dividing SAR target image training set and test set, and generating support set.

Prior to training the twin network, a canonical dataset is required.

1) Firstly, the data set is cut to a uniform size, the SAR target image is cut to a size of 128 x 128, and the image size cannot be uniform by directly using a pooling mode because the imaging principles of the SAR target image and the natural image are different.

2) And then dividing the training set and the testing set, and dividing the data set into two parts of the testing set and the training set. SAR target recognition for an incomplete training set, so the number of samples in the training set is only 50 at most.

3) A small amount of data is then extracted from each class of the training set at equal intervals as a support set. Because the training set and the test set basically comprise SAR images of the same target and different angles, the adoption of equidistant sampling can ensure the diversity of the angles of the samples in the support set. In this way, a training set, a testing set and a supporting set of uniform size are generated.

4) And finally, serializing the training set, the testing set and the supporting set into a file.

The entire dataset specification flow is shown in fig. 1.

And 2, constructing and initializing a twin network.

The twin network is composed of a feature extractor and a discriminator. The feature extractor is a shared-weight two-way convolution network, the two-way convolution network is provided with a left input structure and a right input structure which are the same, the input size is 128 x 128 single-channel gray level pictures, the structure of a first convolution layer is a convolution kernel of 64 6*6, the activation function is a ReLU function, the maximum pooling of 2 x 2 is carried out, the second convolution layer is the same as the first structure, and the activation function and the pooling layer are the same. The third convolution layer had a structure of 128 convolution kernels 3*3, the activation function was still a ReLU function, and the pooling layer was also 2 x 2 max pooling. The fourth convolution layer and the third convolution layer have the same structure, and the activation function and the pooling layer have the same structure. The features extracted by the convolution layer are then expanded into a 1-dimensional tensor, which is further abstracted by the fully-connected layer into a 1-dimensional feature tensor of 4096 length. This 1-dimensional tensor of length 4096 is the last tensor extracted by the feature extractor. After the two-way convolution network extracts the characteristics of the input pair, two characteristic tensors are input into a discriminator, the discriminator firstly obtains the absolute difference of each bit of the two characteristic vectors, then the absolute difference is input into a full-connection layer, the full-connection layer is activated by using a Sigmoid function, and the probability that two input targets are of the same class is output. The structure of the twin network is shown in the following table:

the loss function is cross entropy, the optimizer adopts an Adam optimizer, and the learning rate is 6e-5. Cross entropy is a common concept in deep learning and is generally used to find the difference between a predicted value and a label. The cross entropy is used as a loss function to measure the similarity degree of the predicted value and the label, and the weight W and the bias b are updated by continuous optimization through an optimizer. The cross entropy is represented by the expression (1) as a loss function loss, where y is a label,

for the predicted value, n is the total number of samples for a training batch, i is the sample index from 1 to n.

In contrast to the mean square error (mean squared error, MSE), cross entropy is a convex function that is not prone to falling into local extrema during optimization. The slope of the upper and lower boundaries drops severely when the Sigmoid activation function is used, but the cross entropy is a logarithmic function that still has a higher gradient at the boundary when used as a loss function. This makes the model update faster when the error is larger, avoiding the problem of too long training time.

The Adam optimizer is an optimization method based on a random gradient descent (stochastic gradient descent, SGD) algorithm, combines the advantages of two optimization algorithms, namely adaGrad and RMSProp, and comprehensively considers the first moment estimation and the second moment estimation of the gradient to calculate an update step size. The Adam optimizer can automatically adjust the learning rate, and the working performance is quite excellent under default parameters. The pseudo code of the Adam optimizer is shown in the table below.

Where α is the learning rate or step size, and the weight update rate is controlled. Beta ₁ ,β ₂ The decay rate is the first moment estimate and the second moment estimate. Epsilon is to prevent errors in the calculation that divide by 0. f (θ) is a random objective function. t is the time step.

The structure of the twinning network is constructed only, data can not be directly input, and the constructed network is initialized. The weight W and the bias b are initialized by a random function of Gaussian distribution, wherein the initialization mean value of W is 0, and the standard deviation is 1e-2.b has an initialized mean value of 0.5 and a standard deviation of 1e-2.

And 3, training the twin network.

After the construction and initialization of the twin network are completed, training of the model of the twin network is started. Firstly, loading a training set, a testing set and a supporting set of the serialized SAR image into a video memory. 32 pairs of SAR images are then randomly decimated from the training set as one batch before each iteration, the first 16 pairs of 32 pairs of inputs being the same kind of SAR target and the last 16 pairs of inputs being different kinds of combinations. After obtaining an input pair of a batch, the batch is input to the initialized network to start forward propagation.

When forward propagation is carried out, each image pair is input into a twin network, the convolution layer extracts characteristics, and the input SAR target image is changed into a characteristic map of the SAR target. Wherein each input neuron is multiplied by a weight W during convolution, added with a bias b, and then activated by an activation function. As shown in equation (2).

/>

Wherein the method comprises the steps of

Represents the mth row and nth column outputs of the kth layer convolutional neural network. />

Is the weight corresponding to the output of the kth layer convolution neural network and the mth row and the nth columnA matrix of values. />

Is the input section corresponding to the m-th row and n-th column outputs of the k-th layer convolutional neural network. b ^k Is the bias matrix corresponding to the m-th row, n-th column output of the k-th layer convolutional neural network. f (x) is an activation function, typically a ReLU function and a Sigmoid function.

The activation function typically used after the convolution layer is a ReLU activation function, which as shown in equation (3) changes all negative values to 0 and positive values are unchanged compared to other activation functions. This operation is called unilateral suppression, which makes the calculation simpler, while also giving sparse activation to the neural network. And the ReLU activation function has a wider excitation boundary, so that training of a neural network can be accelerated, and the problem of gradient disappearance does not exist. But if the learning rate is set too high, this can lead to irreversible death of the neurons during training. An appropriate learning rate needs to be set to avoid this problem.

f(x)＝max(0,x)⑶

In the latter two fully connected layers, a Sigmoid function is employed as an activation function, the formula of which is shown in (4).

As the Sigmoid function is used as an activation function, the error is seriously attenuated when in counter propagation along with the increase of the number of layers of the neural network, and finally the gradient disappears and the weight update stagnates. The use of Sigmoid functions is minimized. But the last two fully connected layers represent the absolute differences and the likelihoods of the features, respectively, and the positive integer part after weighting is preserved by using the ReLU activation function at this time is not mapped between the new value ranges (0, 1) as the positive integer part is demapped by using the Sigmoid function. And the probability output by the last layer, the concept of the result of the Sigmoid function can not be strictly equal to the probability, but can be intuitively understood and compared.

The forward propagation finally outputs a predicted value, the predicted value and the true value are used for calculating errors by using a defined loss function, the errors are propagated in the opposite direction, the chain rule is used for calculating partial derivatives of the weights, and then the weights are updated. The formulas of the chain law and the weight update are shown as (5), (6).

The partial derivative of a certain weight to the total error of the final output is obtained through a chain rule in back propagation. This partial derivative is required because the magnitude of the update amount is related to the weight when it is updated later.

In the chain rule, a certain weight w to be updated is obtained _ij For total error E _total After the partial derivative of (2), this partial derivative is multiplied by the learning rate eta, the result being the amount by which the weight needs to be changed. As shown in formula (6), w _ij Subtracting the update amount

A new weight value is obtained.

Setting the iteration times of the training task and the accuracy threshold value of model preservation, and then continuously carrying out iterative training by the twin network to update the weight. The loss function is output every iteration, and the current number of iterations and the loss function are output every 50 iterations. And (3) carrying out verification on the test set once every 200 rounds of iteration are completed, if the accuracy is higher than the threshold, saving the model updating threshold, otherwise, continuing the iteration. Finally training until reaching to meet the iteration stop condition, and storing the optimal model.

And 4, SAR target recognition.

After the training of the twin network is completed, the trained optimal model is obtained, and the model is loaded during testing. In conducting SAR target recognition testing, support focused samples are needed. When a sample in the test set is identified, the sample and all samples in the support set are formed into an image pair. Inputting the image pairs into a trained twin network, and calculating to obtain the similarity degree of the sample and all samples in the support set. And then 5 support samples with the highest similarity probability are selected by using a k-nearest neighbor algorithm, and the category of the test sample is selected according to the category votes of the 5 support samples. The category with the highest ticket number is the sample category, and when the situation that the ticket numbers are the same occurs, the category of the support sample with the highest similarity probability is directly selected as the category of the sample to be identified.

After all samples in the test set are sequentially subjected to the identification step, the identification accuracy is counted and displayed on a command line. The flow chart is shown in fig. 4.

1) Loading the trained model.

2) The sample to be measured and the sample in the support set are formed into an input pair.

3) Input pairs are input into the network with similar results.

4) And taking the class of the support set sample with the highest similarity with the sample to be detected as the class of the sample to be detected, and completing the identification.

Drawings

FIG. 1 is a flow chart of a data specification.

FIG. 2 is a data enhancement schematic.

FIG. 3 is a twin network training flow diagram.

FIG. 4SAR target recognition flow chart.

Detailed Description

The basic flow of the incomplete training set SAR target recognition is shown in fig. 4, and specifically comprises the following steps:

1) SAR target data is classified into two folders of a training set and a testing set, and different types of data have one subfolder under each of the two folders. And then preprocessing the data, and firstly cutting the SAR target image into uniform sizes. And then the data set is distributed, and the number of samples of each class of the training set is 50 at most because of the recognition under the condition of small samples, and meanwhile, the supporting samples are extracted from each class in the training set to serve as supporting sets. Other samples are transferred to the corresponding category folders of the test set. And then the data of the training set, the testing set and the supporting set are unified and serialized into a file which is convenient to read.

2) The establishment and initialization of the twin network.

Firstly, constructing a twin network structure, and prescribing the size of an input image to be 128×128×1. The twin network is a two-way convolution network and is divided into a left path and a right path, and meanwhile, the left path and the right path are weight sharing, so that the two paths have the same structure. The first network layer is a convolution layer, has 64 convolution kernels of 6*6 size, has an activation function of a ReLU function, initializes weights, has no bias term, and performs L2 regularization on the weights. The second layer is a 2 x 2 max pooling layer, with a step size of 2 x 2. The third layer is a convolution layer, which has 64 convolution kernels of 6*6 size, the activation function is also a ReLU function, both weights and bias terms are initialized, and the weights are L2 regularized. The fourth layer is the largest pooled layer with a step size of 2 x 2 and a pooled size of 2 x 2 as the second layer. The fifth layer is a convolution layer containing 128 convolution kernels of 3*3 size, the activation function is a ReLU function, both weights and bias terms are initialized, and the weights are L2 regularized. The sixth layer is also the largest pooling layer and has the same parameters and structure as the previous pooling layer. The seventh and fifth convolution layers have the same structure and parameters. The eighth layer is to expand the entire extracted feature map into a 1-dimensional tensor. The ninth layer is a full-connection layer, the activation function is a Sigmoid function, the weights and the bias items are initialized, the weights are subjected to L2 regularization, and finally a feature vector with the size of 4096 is output.

The above structure is a structure of a left-right convolution network, and the function of the structure is to extract the characteristics of the SAR target image as a characteristic extractor.

And solving the L1 distance for the characteristic values extracted by the two paths of convolutional neural networks. And adding a full connection layer, wherein the activation function is a Sigmoid function, and the output size is 1.

The L1 distance function is added with the full connection layer to be equivalent to a discriminator, and the similarity between the two SAR target images is judged by utilizing the characteristics of the two SAR target images extracted by the two-way convolution network.

The above is the overall structure of the twin network. The initialization of the weights and bias terms uses a random function with a gaussian distribution with a mean of 0.5 and a standard deviation of 1e-2.

3) Training of the twin network.

Setting the size parameter batch_size of batch as 32, setting the maximum iteration number n_iter and the minimum accuracy best of model preservation.

32 input pairs are randomly extracted from the training set before each iteration and trained as one batch, the first 16 groups of the 32 input pairs are SAR target image pairs of different classes, and the last 16 groups are of the same class.

And inputting the data in the batch into the twin network to obtain a predicted value when iteration is carried out, calculating loss, and updating the weight w and the bias b.

And verifying the accuracy of the current model on the test set, if the accuracy is higher than the best, saving the model, updating the best, and continuing iteration, otherwise, directly continuing iteration.

Stopping iteration after the iteration times reach n_iter, and finishing training.

The training process pseudocode for the twin network is as follows:

4) SAR target recognition

And loading the stored model, testing the SAR target recognition effect of the model by using a test set, forming an input pair by using the samples in the test set and the samples in the support set to obtain the similarity degree of each type of samples in the test set and the samples in the support set, and voting by using a k-nearest neighbor algorithm to obtain the type most similar to the sample to be recognized. And if the number of the tickets is the same, selecting the category with the highest similarity as the category of the sample to be identified.

Comparing the recognized result with the true value, counting the accuracy and displaying in the command line.

Claims (2)

1. The SAR target recognition method based on the incomplete training set of the twin network is characterized by comprising the following steps of: the implementation flow of the method is as follows:

step 1, specification of data sets: unified size, dividing SAR target image training set and test set, and generating support set;

before training the twin network, a canonical dataset is required;

1) Firstly, cutting a data set to a uniform size, and uniformly cutting an SAR target image to 128 x 128 size;

2) Dividing the data set into a test set and a training set;

3) Extracting a small amount of data from each category in the training set at equal intervals as a support set;

4) Finally, serializing the training set, the testing set and the supporting set into a file;

step 2, constructing and initializing a twin network;

the twin network consists of a feature extractor and a discriminator; the feature extractor is a double-path convolution network sharing weight, the double-path convolution network is provided with a left input structure and a right input structure which are the same, the input size is 128 x 128 single-channel gray level pictures, the structure of a first convolution layer is a convolution kernel of 64 6*6, the activation function is a ReLU function, the maximum pooling of 2 x 2 is carried out, the second convolution layer is the same as the first structure, and the activation function and the pooling layer are the same; the third convolution layer has a structure of 128 convolution kernels 3*3, the activation function is still a ReLU function, and the pooling layer is also the maximum pooling of 2 x 2; the fourth convolution layer and the third convolution layer have the same structure, and the activation function and the pooling layer have the same structure; then, the features extracted by the convolution layer are unfolded into a 1-dimensional tensor, and the tensor is further abstracted into a 1-dimensional feature tensor with the length of 4096 by the full connection layer; this 1-dimensional tensor of length 4096 is the last tensor extracted by the feature extractor; after the two-way convolution network extracts the characteristics of the input pair, inputting the two characteristic tensors into a discriminator, firstly solving the absolute value of the difference of each bit of the two characteristic vectors by the discriminator, then inputting the absolute value of the difference into a full-connection layer, activating the full-connection layer by using a Sigmoid function, and outputting the probability that the two input targets are of the same class; the structure of the twin network is shown in the following table:

/>

the loss function is cross entropy, the optimizer adopts an Adam optimizer, and the learning rate is 6e-5; the cross entropy is used as a loss function to measure the similarity degree of the predicted value and the label, and the weight W and the bias b are updated by continuous optimization through an optimizer; the cross entropy is represented by the expression (1) as a loss function loss, where y is a label,

n is the total sample amount of one training batch, i is the sample index from 1 to n;

initializing a network; the weight W and the bias b are initialized by a random function of Gaussian distribution, wherein the initialization mean value of the W is 0, and the standard deviation is 1e-2; b has an initialization mean value of 0.5 and a standard deviation of 1e-2;

step 3, training a twin network;

firstly, loading a training set, a testing set and a supporting set of the serialized SAR image into a video memory; then randomly decimating 32 pairs of SAR image pairs from the training set before each iteration to be used as a batch, wherein the first 16 pairs of 32 pairs of inputs are SAR targets of the same kind, and the second 16 pairs of inputs are combinations of different kinds; after an input pair of a batch is obtained, the batch is input to the initialized network to start forward propagation;

when forward propagation is carried out, inputting each image pair into a twin network, extracting features by a convolution layer, and changing an input SAR target image into a feature map of an SAR target; wherein each input neuron is multiplied by a weight W in the convolution process, added with a bias b, and then activated by an activation function; as shown in formula (2);

wherein the method comprises the steps of

An mth row and an nth column output representing a kth layer convolutional neural network; w (W) ^k Is the weight matrix of the convolutional neural network of the k layer; x is x ^k Is the input part of the k-th layer convolutional neural network; b _k Is a bias matrix for convolving the neural network with the k-th layer; f is an activation function, which is a ReLU function or a Sigmoid function; m and n represent the m-th row and n-th column of the convolutional neural network;

the activation function used after the convolution layer is a ReLU activation function, the formula of which is shown in (3), the ReLU activation function changes all negative values into 0, and the positive values are unchanged;

f(x)＝max(0,x) ⑶

in the two latter fully connected layers, a Sigmoid function is adopted as an activation function, and the formula of the Sigmoid function is shown as (4);

the forward propagation finally outputs a predicted value, the predicted value and the true value are used for calculating errors by using a defined loss function, then the errors are propagated reversely, the chain rule is used for calculating partial derivatives of weights, and then each weight is updated; the formulas of the chain rule and the weight update are shown as (5) and (6);

in the back propagation, the partial derivative of a certain weight to the total error output finally is obtained through a chain rule;

in the chain rule, a certain weight w to be updated is obtained _ij For total error E _total After the partial derivative of (2), multiplying the partial derivative by the learning rate eta, and obtaining a result which is the amount of the weight to be changed;

setting iteration times of a training task and an accuracy threshold value of model preservation, and then continuously carrying out iterative training by a twin network to update weights; outputting a loss function every iteration, and outputting the current iteration times and the loss function every 50 iterations; performing verification on the test set once every 200 rounds of iteration are completed, if the accuracy is higher than the threshold, saving the model updating threshold, otherwise, continuing iteration; finally training until reaching the condition of meeting the iteration stop, and storing an optimal model;

step 4, SAR target recognition;

after the training of the twin network is completed, obtaining a trained optimal model, and loading the model during testing; when SAR target recognition test is carried out, a support concentrated sample is needed; when a certain sample in the test set is identified, the sample and all samples in the support set are formed into an image pair; inputting the image pairs into a trained twin network, and calculating to obtain the similarity degree of the sample and all samples in the support set; then 5 support samples with the highest similarity probability are selected by using a k-nearest neighbor algorithm, and the category of the test sample is selected according to the category votes of the 5 support samples; the category with the highest ticket number is the sample category, and when the situation that the ticket numbers are the same occurs, the category of the support sample with the highest similarity probability is directly selected as the category of the sample to be identified.

2. The SAR target identification method of the incomplete training set based on the twinning network according to claim 1, wherein: after all samples in the test set are sequentially subjected to the identification step, the identification accuracy is counted and displayed in a command line;

1) Loading a model with training completed;

2) Forming an input pair by a sample to be measured and a sample in a support set;

3) Inputting the input pair into the network to obtain a similar result;

4) And taking the class of the support set sample with the highest similarity with the sample to be detected as the class of the sample to be detected, and completing the identification.

Images