CN110516596A - Spatial Spectral Attention Hyperspectral Image Classification Method Based on Octave Convolution - Google Patents

Abstract

The invention discloses a space-spectral attention hyperspectral image classification method based on Octave convolution, which solves the problems in the prior art that the distance between the same category is large, the distance between different categories is small, and the classification accuracy is low. The plan is: image input to be classified and data preprocessing, division of training set and test set, construction of Octave convolutional neural network, determination of Octave convolutional neural network loss function, training update of Octave convolutional neural network, test set data testing, completed Hyperspectral Image Classification. The present invention uses Octave convolution operation to strengthen feature representation, introduces spatial attention mechanism and spectral attention mechanism, and enables the network to more accurately find regions that are more favorable for classification and contain more comprehensive and detailed information. The invention has high classification accuracy and strong robustness, and can be applied to the analysis and management of hyperspectral image data.

Description

Spatial Spectral Attention Hyperspectral Image Classification Method Based on Octave Convolution

technical field

The invention belongs to the technical field of image processing, and in particular relates to content classification of hyperspectral images, in particular to a spatial spectral attention hyperspectral image classification method based on Octave convolution, which can be applied to the analysis and management of hyperspectral image data.

Background technique

With the continuous improvement of the pixel resolution of hyperspectral images, more useful data and information can be obtained from hyperspectral images. According to the needs of different applications, the processing of hyperspectral images also has different requirements, so in order to effectively analyze and manage these hyperspectral image data, it is necessary to attach a semantic label to each pixel of the hyperspectral image. Hyperspectral image classification is an important way to solve this kind of problem. Hyperspectral image classification refers to distinguishing pixels with similar characteristics from a hyperspectral image, and correctly classifying these pixels. Compared with natural images, hyperspectral images have their own characteristics. Due to the limitation of spatial resolution of hyperspectral images and the phenomenon of the same object with different spectra and different objects with the same spectrum, the classification results often cause misclassification. phenomenon, which is caused by the complexity of the hyperspectral imagery itself. Therefore, how to classify hyperspectral images more accurately has become a great challenge at present.

The classification based on convolutional neural network refers to inputting some data that needs to be trained into the convolutional neural network in batches. Through repeated training of large batches of data, the target optimization loss function is continuously reduced, so as to achieve the purpose of classification. . Nowadays, many mature and well-known convolutional neural networks have been proposed. For example, the deep residual convolutional neural network for image classification tasks proposed by He Kaiming and others in 2015 is widely used by everyone. The deep residual convolutional neural network effectively solves the problem that the features extracted by the image classification task contain insufficient information and the network training gradient disappears. However, this network still has large distances between the same category and small distances between different categories due to the complexity of the image data. , the problem of low image classification accuracy, and the low robustness of the network caused by the small number of training samples, which is easy to fall into the problem of over-fitting.

Although the existing convolutional neural network can achieve the task of hyperspectral pixel-level classification, there are still three deficiencies in learning image semantic information: first, the classification information location is not accurate due to the complexity of hyperspectral images, and the When classifying tasks, the distance between the same category is large, and the distance between different categories is small; secondly, when extracting hyperspectral features, the insufficient utilization of the extracted features causes information loss or retains too much irrelevant information, resulting in information redundancy, which affects At the same time, the convolutional neural network often falls into the local optimal area during training; the third is that there are relatively few hyperspectral data available, and training convolutional neural networks usually requires a large amount of training data, and a small amount of hyperspectral data cannot satisfy volume. The data requirements of the product neural network. These three deficiencies will lead to problems of poor robustness and misclassification in the classification process of actual hyperspectral images.

Contents of the invention

The purpose of the present invention is to solve the problems in the above-mentioned prior art, and propose a hyperspectral image classification method based on deep learning of Octave convolution space-spectral attention mechanism with higher classification accuracy.

The present invention is a kind of hyperspectral image classification method based on Octave convolution spatial spectrum attention mechanism depth learning, is characterized in that, comprises the following steps:

(1) Image input and data preprocessing: input the hyperspectral image to be classified, and slide pixel by pixel with each pixel as the center, and all the image blocks obtained by sliding are used to establish the hyperspectral image library {I ₁ ,I ₂ ,…,I _n ,…,I _N }, the category corresponding to each image block in the image library is {Y ₁ ,Y ₂ ,…,Y _n ,…,Y _N }, and the established hyperspectral image library is Normalization processing, where I _n represents the nth image block in the image library, Y _n represents the category corresponding to the nth image block in the image library, n represents the nth sample number in the image library, n∈[0,N] , N represents the total number of image blocks in the hyperspectral image database;

(2) Divide training set and test set: randomly select a specified number of hyperspectral image samples from each type of hyperspectral image after normalization processing, and construct a training sample set {T ₁ ,T ₂ ,…,T _j ,… ,T _M }, take the remaining hyperspectral image as the test sample set {t ₁ ,t ₂ ,…t _d ,…,t _m }, where T _j represents the jth sample in the training sample, j∈[0,M ], t _d represents the dth sample in the test sample, d∈[0,m], M is the total number of training samples, m is the total number of test samples, m<N, M<N;

(3) Octave convolutional neural network construction: build an Octave convolutional neural network, the input end of the network is the Octave convolution module, the output end of the network is the output result of the fully connected layer, between the input end and the output end of the network Contains two branches, one of which passes through the spatial attention module and the pixel-level attention module in turn, and the other branch passes through the spectral attention module and the pixel-level attention module in turn;

(4) Determine the loss function loss _op of the Octave convolutional neural network: setting the loss function includes inputting the extracted features after feature fusion to the fully connected layer and then obtaining the cross-entropy loss between the output classification result and the actual result ₁ , the spatial attention The features extracted by the force module and the pixel-level attention module are input to the fully connected layer to obtain the cross-entropy loss between the output classification result and the actual result _2. The features extracted by the spectral attention module and the pixel-level attention module are input to the fully connected The cross-entropy loss ₃ of the output classification result and the actual result obtained by the layer and the L2 norm of the weight W of the convolutional neural network with hyperparameters are four parts. The loss function of the network is composed of the above four parts in sequence;

(5) Training update: Set the number of iterations of network training to P, and iteratively train the Octave convolutional neural network through gradient descent optimization until the loss function loss _op does not decrease or the number of training rounds reaches the number of iterations, and the trained Octave volume is obtained product neural network;

(6) Data test: Input the normalized test sample set into the trained Octave convolutional neural network to obtain the classification result and complete the image classification.

The present invention adopts the Octave convolution operation and integrates the attention mechanism, and sets a deep convolution network model including the Octave convolution operation and the attention mechanism. This model only needs a small amount of training data to train the effect better The Octave convolution operation effectively combines the high-frequency part and low-frequency part of the hyperspectral image data. The extracted features contain more comprehensive and detailed information, and the utilization rate is higher. At the same time, the attention mechanism can make the network faster Effectively find feature regions that are more favorable for classification tasks, make the information captured by the network more comprehensive and accurate, and effectively solve the problems of low classification accuracy and weak robustness in current hyperspectral image classification tasks.

Compared with the prior art, the present invention has the following advantages:

Feature representation enhancement: the present invention introduces the Octave convolution operation into the hyperspectral image classification method for the first time, and the Octave convolution module is specially set in the Octave convolution neural network, because it is based on the Octave convolution operation to obtain the hyperspectral image features , the high-frequency information and low-frequency information of the hyperspectral image are combined reasonably and effectively, so that the information contained in the hyperspectral image features is more comprehensive and detailed, and the feature representation of the image is enhanced.

Improved classification accuracy: the present invention introduces the attention mechanism into the hyperspectral image classification method. The Octave convolutional neural network is specially equipped with a spatial attention mechanism module, a spectral attention mechanism module and a pixel-level attention mechanism module. Due to the introduction of the principle of attention mechanism, the network can quickly and accurately find the most obvious region among the acquired hyperspectral image features, so that the features that are more beneficial to classification are more concentrated in a certain region with obvious semantic information, reducing the loss. The probability that the function falls into a local optimum enhances the accuracy of hyperspectral image classification.

Robustness enhancement: The present invention designs a more effective loss function. The new loss function uses three cross-entropy loss functions to promote the network to learn more effective features for hyperspectral image classification, which strengthens the feature representation of images and further clarifies The classification task is more purposeful, and it can adapt to complex hyperspectral image data, which greatly enhances the robustness of the network.

Less training samples: The present invention only needs a small number of samples to train a network model with better effect, and requires less data volume of hyperspectral images.

Description of drawings

Fig. 1 is the realization flowchart of the present invention;

Fig. 2 is the Octave convolutional neural network structural diagram constructed among the present invention;

Fig. 3 is the hyperspectral image used in experiments in the present invention, wherein Fig. 3 (a) is the original hyperspectral image, and Fig. 3 (b) is the class label of the corresponding pixel of the original hyperspectral image;

Figure 4 is a structural diagram of the Octave convolution module;

Figure 5 is a structural diagram of the spatial attention mechanism module;

Figure 6 is a block diagram of the inter-spectrum attention mechanism;

Figure 7 is a block diagram of the pixel-level attention mechanism.

Detailed ways

The technical solutions and effects of the present invention will be described in detail below in conjunction with the accompanying drawings.

Example 1

In recent decades, hyperspectral resolution has provided useful information for distinguishing different materials and objects, and hyperspectral image classification methods have been widely used in earth observation, especially in urban development, precision agriculture, land change detection, resource Management etc. are of great significance. In the existing methods of hyperspectral image classification, see Figure 1, after constructing hyperspectral image library, dividing training sample set and test sample set, building and training convolutional neural network model, and performing training on the trained convolutional neural network model The classification of hyperspectral images has been completed in tests, but due to the complexity of hyperspectral images, the location of classification information is inaccurate, and the trained network is easy to fall into the problem of local optimum; at the same time, due to the deep network depth and complex network operation , the extracted features contain insufficient information, which leads to the problem that the distance between the same category is large and the distance between different categories is small when performing classification tasks, which is prone to misclassification and poor robustness.

In view of this current situation, the present invention has carried out research and discussion, and proposes a hyperspectral image classification method based on deep learning of Octave convolution space-spectral attention mechanism, see Figure 2, including the following steps:

(1) Image input and data preprocessing: input the hyperspectral image to be classified. The hyperspectral image has many spectral bands, and the difference between different bands is large. At the same time, it is affected by environmental factors such as light and temperature, and belongs to the same category of hyperspectral The images are quite different, and the hyperspectral images belonging to different categories have little difference; for the hyperspectral image data, each pixel is centered on the hyperspectral image to slide pixel by pixel. In this example, the size of 9*9 is selected. Sliding frame, the size of the sliding frame can be adjusted according to the actual situation. Use the sliding frame to slide pixel by pixel. The distance of each sliding is one pixel. Each sliding can get a hyperspectral image block with a size of 9*9. All the obtained hyperspectral image blocks are used to establish the hyperspectral image library {I ₁ ,I ₂ ,…,I _n ,…,I _N }, and the categories corresponding to each image block in the image library are respectively {Y ₁ ,Y ₂ ,…,Y _n ,…,Y _N }, for the established hyperspectral image library, find the maximum and minimum values of all pixels in the image library, using the values of all pixels and the most value pair of two pixels The established hyperspectral image library is normalized, where In represents the _{nth image in the image library, Y n represents} the category corresponding to the nth image in the image library, n represents the nth sample number in the image library, and n ∈[0,N], N represents the total number of pictures in the hyperspectral image database.

(2) Divide training set and test set: randomly select a specified number of hyperspectral image samples from the normalized hyperspectral images of each category, and construct a training sample set {T ₁ ,T ₂ ,…,T _j ,…,T _M }, referred to as the training set, the remaining hyperspectral images are used as the test sample set {t ₁ ,t ₂ ,…t _d ,…,t _m } This test sample set is a normalized test sample set, referred to as the test set. T _j in the training sample set represents the jth sample in the training sample, j∈[0,M], t _d in the test sample set represents the dth sample in the test sample, d∈[0,m], M is the total number of training samples number, m is the total number of test samples, m<N, M<N.

When the present invention selects samples to build a training sample set, it adopts a random selection method for each category, so that the training sample set selected in this way can contain samples of all categories, and at the same time maximize all possible samples contained in each category. are divided into the training sample set.

When constructing a hyperspectral image training sample set, it is usually a routine practice for those skilled in the art to randomly select a specified number of training samples, mainly because in hyperspectral image data, there are few hyperspectral image samples and the number of samples of each category is relatively different. Large, if the training sample set and test sample set are divided proportionally, the classification accuracy of the hyperspectral category with few samples will be very low.

(3) Construction of Octave convolutional neural network: build an Octave convolutional neural network, see Figure 2, the input end of the network is the Octave convolution module, the output end of the network is the output result of the fully connected layer, and the input end of the network is connected with There are two branches between the output terminals, one of which passes through the spatial attention module and the pixel-level attention module in turn, and the other branch passes through the spectral attention module and the pixel-level attention module in turn.

The Octave convolution module of the present invention fully considers the high-frequency information and low-frequency information of the hyperspectral image, effectively combines the high-frequency information and low-frequency information of the hyperspectral image through the convolution operation, enhances the feature representation of the image, and makes the image The information contained in the feature is more comprehensive; the spatial attention module and the pixel-level attention module make the whole network focus on a certain area with obvious semantic information, so that the whole network can make full use of the most important information for classification in the process of hyperspectral classification. Favorable areas, grasp the most obvious feature areas, improve the classification accuracy of hyperspectral; spectral attention module and pixel-level attention module make the whole network focus on the most useful spectral bands for hyperspectral image classification, highlighting The feature represents a better spectral band, which is conducive to improving the accuracy of hyperspectral classification and improving the robustness of the network.

(4) Determine the loss function loss _op of the Octave convolutional neural network: setting the loss function includes inputting the extracted features after feature fusion to the fully connected layer and then obtaining the cross-entropy loss between the output classification result and the actual result ₁ , the spatial attention The features extracted by the force module and the pixel-level attention module are input to the fully connected layer to obtain the cross-entropy loss between the output classification result and the actual result _2. The features extracted by the spectral attention module and the pixel-level attention module are input to the fully connected The cross-entropy loss ₃ of the output classification result obtained by the layer and the actual result and the L2 norm of the weight W of the convolutional neural network with hyperparameters are four parts. The loss function of the network is formed by adding the above four parts in sequence.

The loss function loss _op of the present invention uses three cross-entropy functions to promote the network to learn more effective features for hyperspectral image classification, strengthens the feature representation of hyperspectral images, further clarifies the classification task, has stronger purpose, and can adapt to complex Hyperspectral image data greatly enhances the robustness of the network, and at the same time effectively reduces the probability of the loss function falling into local optimum during the network training process.

(5) Training update: Set the number of iterations of network training to P, and iteratively train the Octave convolutional neural network through gradient descent optimization until the loss function loss _op does not decrease or the number of training rounds reaches the number of iterations, and the trained Octave volume is obtained product neural network. The number of iterations P is set artificially and can be adjusted according to the training effect of the network to make the classification accuracy of hyperspectral images the highest. In the process of network training, the learning rate of the network gradually decreases with the training of the network. The learning rate is relatively large during the initial training. As the training deepens, the learning rate gradually decreases, so that the network effectively reduces the loss function of the network. The local optimal probability is beneficial to the improvement of hyperspectral image classification accuracy and robustness.

(6) Data test: Input the normalized test sample set into the trained Octave convolutional neural network to obtain the classification result and complete the image classification.

The present invention provides an overall technical scheme of a hyperspectral image classification method based on deep learning of Octave convolution space-spectral attention mechanism.

The technical idea of the present invention is: to build an Octave convolutional neural network, and use Octave convolution operation to obtain more comprehensive convolution features for each pixel area; according to the principles of spatial attention mechanism and pixel-level attention mechanism , focus the attention of the entire network on an area with obvious semantic information, and find an area with useful information that is conducive to classification; Strong spectral bands to obtain stronger feature representations that are conducive to classification; through the above attention mechanism, the attention of the entire network is focused on the most effective area for classification, and then image classification is achieved through a fully connected layer network.

The present invention solves the problems existing in the current hyperspectral image classification, such as large distance between the same category, small distance between different categories, low image classification accuracy, low network robustness and easy to fall into over-fitting problems caused by few training samples. question.

The invention effectively obtains more comprehensive convolution features containing information through the Octave convolution module, enhances the feature representation of the image, and uses the spatial attention module and the pixel-level attention module to enable the network to find more obvious semantic information and prominent features. region, using the spectral attention module and pixel-level attention module to find spectral bands with stronger spectral information, and through the combination of several attention mechanisms, find more effective feature regions for hyperspectral image classification, improving the performance of hyperspectral image classification Accuracy increases the robustness of the network.

Example 2

The hyperspectral image classification method based on the deep learning of the spatial spectrum attention mechanism of Octave convolution is the same as embodiment 1, and the Octave convolution neural network described in step (3) of the present invention is built, referring to Fig. 2, the present invention forms Octave convolution The Octave convolution module, spatial attention module, spectral attention module, pixel-level attention module and fully connected layer of the neural network, the parameters of each component module are set as follows:

The Octave convolution module is the input module, which consists of four convolution parts connected in sequence, and each convolution part includes Octave convolution. See Figure 4, Batch Normalization and Relu activation functions, in the second and third convolutions There is also a max pooling layer between parts.

The Octave convolution operation mainly divides the hyperspectral image into two parts, the high-frequency part and the low-frequency part, and then convolves the high-frequency part and the low-frequency part through the conventional convolution operation to obtain the image from high frequency to high frequency and high frequency to low frequency. The four convolution results of low frequency, low frequency to high frequency, and low frequency to low frequency are added together, and the convolution results of high frequency to high frequency and low frequency to high frequency are added together, and the convolution results of high frequency to low frequency and low frequency to low frequency are compared Added together, the high-frequency information and low-frequency information of the hyperspectral image are effectively connected and communicated, so that the obtained feature representation contains more comprehensive and detailed information, and the feature representation of the hyperspectral image is enhanced.

The spatial attention mechanism module, see Figure 5, consists of a convolutional layer, Batch Normalization, Relu activation function, matrix transposition and multiplication layer, softmax layer, and data transposition and addition layer. The features input to the spatial attention mechanism module are strengthened by a convolution operation, input to the matrix transposition and multiplication layer, and a matrix is obtained, and each element in the matrix represents the space of any two positions in the hyperspectral image relationship; then use the softmax layer to normalize the matrix, normalize the values in the matrix to between 0 and 1, and multiply the normalized matrix with the features obtained after the convolution operation, Highlight a region with obvious semantic information in the hyperspectral image, and obtain a feature map with obvious semantic information representation; finally, the features with obvious semantic information and the initial features input to the spatial attention mechanism module are input to the data transposition and correlation In the addition layer, the two features are added together to prevent the loss of information. The final feature semantic information obtained by the spatial attention mechanism module is obvious, which makes it easy for the network to find the region with the most obvious features, and improves the accuracy of hyperspectral image classification.

The spectral attention mechanism module, see Figure 6, consists of a matrix transposition and multiplication layer, a softmax layer, and a data transposition and addition layer. The features input to the spectral attention mechanism module are input to the matrix transposition and multiplication layer to obtain a matrix, and each element in the matrix represents the relationship between different spectral bands of any pixel in the hyperspectral image; then use the softmax layer Normalize this matrix, normalize the values in the matrix to between 0 and 1, and multiply the normalized matrix with the initial features input to the spectral attention module to highlight the hyperspectral image. The spectral band with the strongest spectral information in each pixel is obtained to obtain a feature map that highlights the strongest spectral information; finally, the features that highlight the strongest spectral information and the initial features input to the spectral attention module are input to the data transposition and addition layer, adding two features together to prevent loss of information. The final feature obtained by the spectral attention mechanism module highlights the spectral band with the strongest spectral information, making the network focus on the region with the strongest spectral information, improving the accuracy of hyperspectral image classification and enhancing the robustness of the network. sex.

The pixel-level attention mechanism module, see Figure 7, consists of convolutional layers, Batch Normalization and Relu activation functions. Hyperspectral image classification is to classify each pixel in the hyperspectral image. The features obtained by the pixel-level attention mechanism module further refine the features of each pixel, enhance the feature representation of each pixel, and improve the Accuracy of Hyperspectral Image Classification.

The fully connected layer is composed of the first fully connected layer, the second fully connected layer and the softmax layer, that is, the output layer. The input of the first fully connected layer is the features obtained after the spatial attention module and the pixel-level attention module, the features obtained after the spectral attention module and the pixel-level attention module, and the features obtained after the two kinds of features are added and fused. Fusion features; the input of the second fully connected layer is the output feature of the first fully connected layer; the input of the softmax layer is the output feature of the second fully connected layer, and the output result of the softmax layer indicates that the training sample belongs to a certain category in the hyperspectrum Probability, the output of the softmax layer is the final output of the entire network.

The present invention uses Octave convolution operation to strengthen feature representation, introduces spatial attention mechanism and spectral attention mechanism, enables the network to more accurately find areas that are more beneficial to classification and contain more comprehensive and detailed information, and enhances the accuracy of hyperspectral image classification and network robustness.

Example 3

The hyperspectral image classification method based on the deep learning of the spatial spectrum attention mechanism of Octave convolution is the same as embodiment 1-2, and the determination of the Octave convolution neural network loss function loss _op as described in step 4 specifically includes the following steps:

(4a) Input the training image library {T ₁ ,T ₂ ,…,T _j ,…,T _M } into the Octave convolution module of the Octave convolutional neural network, and output the feature F of the last layer of the convolutional layer.

(4b) Input the last layer feature F into the spatial attention module and spectral attention module of the Octave convolutional neural network, the output features are A and B respectively, and then input the output features A and B to the pixel-level attention module , the output features are C and D respectively.

(4c) The obtained features C and D are input to the fully connected layer of the Octave convolutional neural network, and the output classification results obtained by using the features C and D are output; at the same time, the features C and D are multiplied by a coefficient respectively, and then step by step The pixels are added to obtain the fused feature E, and then the feature E is input to the fully connected layer of the Octave convolutional neural network, and the output classification result obtained by using the fused feature E is output, and the loss function loss of the Octave convolutional neural network is obtained _op :

Among them, loss ₁ is the cross entropy between the output classification result and the actual result after using the fused feature E through the fully connected layer, loss ₂ is the cross entropy between the output classification result and the actual result after the feature C passes through the fully connected layer, and loss ₃ is the feature D outputs the cross entropy of the classification result and the actual result after passing through the fully connected layer, is the L2 norm of the convolutional neural network weight vector, η is hyperparameters.

The loss function loss _op of the present invention uses three cross-entropy functions to promote the network to learn more effective spatial and spectral features for hyperspectral image classification, strengthen the feature representation of hyperspectral images, further clarify the classification task, and have a stronger purpose. It can adapt to complex hyperspectral image data, greatly enhances the robustness of the network, and effectively reduces the probability of the loss function falling into local optimum during the network training process.

Example 4

The hyperspectral image classification method based on the deep learning of the space-spectrum attention mechanism of Octave convolution is the same as the cross-entropy loss ₁ of the output classification result in (4c) and the actual result in Embodiment 1-3, and the formula is as follows:

Among them, y _j is the predicted class label probability of T _j in the training image library, o _j is the actual class label of T _j in the training image library; the input of loss ₁ is the predicted class obtained after the fused feature E passes through the fully connected layer mark probability;

The principles of loss ₂ and loss ₃ in the present invention are the same as the formula expression and loss ₁ , except that the input of loss ₂ is the predicted class label probability obtained after the feature C passes through the fully connected layer, and the input of loss ₃ is the feature D after passing through the fully connected layer The resulting predicted class label probabilities.

Example 5

The hyperspectral image classification method based on the deep learning of the spatial spectrum attention mechanism of Octave convolution is the same as embodiment 1-4, and in step (1), the hyperspectral image database is normalized, and is carried out by the following formula:

Where V _max is the point maximum value of all pixels in the hyperspectral image library, V _min is the point minimum value of all pixels in the hyperspectral image library, V _n is the pixel value of any point in the hyperspectral image library, {I′ ₁ , I′ ₂ ,…,I′ _n ,…,I′ _N } is the hyperspectral image database after normalization processing, I′ _n is the nth sample of hyperspectral image after normalization processing, n∈[0, N].

The present invention limits the hyperspectral pixel value to -0.5 to 0.5 by normalizing the hyperspectral image database, so that the brightness distribution of the hyperspectral image is more balanced, effectively avoiding the interference caused by subsequent processing, and at the same time, each The pixel value of the pixel is limited to a uniform interval to prevent the pixel value from being large and erasing the edge information. Due to the normalization, the pixel value of the hyperspectral image is reduced, which reduces the computational load of the network and accelerates the convergence of the network training. Experiments also prove that normalizing hyperspectral image pixel values between -0.5 and 0.5 further improves the accuracy of hyperspectral image classification, and at the same time the network training speed is greatly accelerated.

Example 6

The hyperspectral image classification method based on the deep learning of the spatial spectrum attention mechanism of Octave convolution is the same as embodiment 1-5, and in step (5), the convolutional neural network is iteratively trained by gradient descent optimization, and its realization is as follows:

(5a) Set the initial learning rate of training to L and the decay rate to β, and divide the training image library {T ₁ , T ₂ ,…,T _j ,…,T _M } into the convolutional neural network constructed by G times of input , the number of pictures input each time is Q, then:

Where M is the total number of samples in the training image database.

(5b) Let the learning rate l corresponding to each input picture be:

l= ^L *βG

(5c) update the parameters of the convolutional neural network G times by the following formula, and obtain the updated weight vector W _new ;

Among them, W is the weight vector of convolutional neural network parameters;

(5d) Input the next training picture into the convolutional neural network, and update the loss function loss _op after the weight vector is updated, so that the value of the loss function loss _op continues to decrease;

(5e) Repeat (5d) until the loss function loss _op no longer decreases, and the current number of training rounds is less than the set iteration number P, then stop the training of the network to obtain a trained convolutional neural network; otherwise, when training When the round reaches the set number of iterations P, the training of the network is stopped, and the trained convolutional neural network is obtained.

The number Q of pictures input each time in the present invention is artificially set, and can be adjusted according to the training effect of the network, so that the classification accuracy of hyperspectral images is the highest. The learning rate of the network is the rate at which the network learns effective features. During the network training process, the learning rate of the network gradually decreases with the training of the network. The learning rate is relatively large during the initial training, which promotes the network to quickly and efficiently learn the hyperspectral image. The main features, as the training deepens, the learning rate gradually decreases, and the network learning speed slows down, which prompts the network to learn detailed features that are beneficial to hyperspectral image classification, and at the same time effectively reduces the network's loss function into a local optimum. The probability speeds up the speed of network training and accelerates the convergence of network training.

A more detailed example is given below to further illustrate the present invention:

Example 7

The hyperspectral image classification method based on the spatial spectrum attention mechanism deep learning of Octave convolution is the same as implementing 1-6, referring to Fig. 2, the implementation steps of the present invention are as follows:

Step 1, establish a hyperspectral image library, and obtain training samples and test samples.

1a) Download the Indian Pines hyperspectral image dataset from the relevant official website. The Indian Pines hyperspectral image dataset was imaged by the Airborne Visible Infrared Imaging Spectrometer (AVIRIS) in 1992 on an Indian pine tree in Indiana, USA, see Figure 3, Figure 3(a) is the original Indian Pines hyperspectral image, and Figure 3(b) is the class label of the corresponding pixel of the original Indian Pines hyperspectral image. Swipe pixel by pixel with each pixel as the center on the hyperspectral image, select a sliding frame with a size of 13*13, and use the sliding frame to slide pixel by pixel. The distance of each sliding is one pixel, and each sliding is A hyperspectral image block with a size of 13*13 can be obtained, and all the hyperspectral image blocks obtained by sliding are used to establish a hyperspectral image library {I ₁ ,I ₂ ,…,I _n ,…,I _N }, and the image library corresponds to The category is {Y ₁ , Y ₂ ,...,Y _n ,...,Y _N }, where I _n represents the nth image in the image library, Y _n represents the category corresponding to the nth image in the image library, and n represents the image library In the nth sample number, n∈[0,N].

1b) For the established hyperspectral image library, find the maximum and minimum values of all pixels in the image library, and use the values of all pixels and the maximum value of two pixels to establish a hyperspectral image library according to the following formula Normalization processing:

Where V _max is the point maximum value of all pixels in the hyperspectral image library, V _min is the point minimum value of all pixels in the hyperspectral image library, V _n is the pixel value of any point in the hyperspectral image library, {I′ ₁ , I′ ₂ ,…,I′ _n ,…,I′ _N } is the hyperspectral image library after normalization processing, I′ _n is the nth sample of remote sensing image after normalization processing, n∈[0, N].

1c) Randomly select a specified number of hyperspectral image samples from the normalized hyperspectral images of each category to construct a training sample set {T ₁ ,T ₂ ,…,T _j ,…,T _M }, referred to as The training set, the remaining hyperspectral images are used as the test sample set {t ₁ ,t ₂ ,…t _d ,…,t _m } referred to as the test set, where T _j represents the jth sample in the training sample, j∈[0, M], t _d represents the dth sample in the test sample, d∈[0,m], M is the total number of training samples, m is the total number of test samples, m<N, M<N.

Step 2, construct the Octave convolutional neural network.

2a) Set the Octave convolution module, which consists of four convolution parts connected in sequence, each convolution part includes Octave convolution, Batch Normalization and Relu activation function, between the second and third convolution parts There is a max pooling layer;

Referring to Fig. 4, the working principle of Octave convolution of the present invention is as follows:

Divide the image into two parts, high frequency and low frequency, where the width and height of the image in the low frequency part are half of the high frequency part; perform ordinary convolution operations on the high frequency part, and obtain two convolution results, in which the high frequency to The width and height of the high-frequency convolution result are the same as those of the high-frequency part, and the width and height of the high-frequency to low-frequency convolution result are the same as those of the low-frequency part; then perform the same convolution operation on the low-frequency part, where the low-frequency to high-frequency convolution The width and height of the convolution result are the same as the high-frequency part, and the width and height of the low-frequency to low-frequency convolution result are the same as those of the low-frequency part; then the results with the same width and height are added together to form a new high-frequency part and low-frequency part .

The Relu activation function is:

Where x is the input function of the Relu activation function.

2b) Set the spatial attention mechanism module, which consists of convolutional layer, Batch Normalization, Relu activation function, matrix transposition and multiplication layer, softmax layer, and data transposition and addition layer. Its structure is shown in Figure 5.

In the matrix transposition and multiplication layer in the spatial attention module of the present invention, the input feature is the feature obtained through convolution, and its size is W×H×C. First, the feature size is converted to N×C, where N=W× H, and then perform matrix transposition on the converted features, the size of the obtained features is C×N, and then perform matrix multiplication on the features obtained through feature conversion and the features obtained through matrix transposition to obtain the output matrix, its size It is N×N.

The softmax layer in the spatial attention module of the present invention uses softmax to normalize the matrix output from the matrix transposition and multiplication layer, normalizes the values in the matrix to between 0 and 1, and simultaneously normalizes The matrix of is multiplied by the features obtained through the convolution operation and matrix transformation to obtain the output features, whose size is N×C.

In the data transposition and addition layer in the spatial attention module of the present invention, the output result of the softmax layer is converted first, and the size after conversion is W×H×C, and then the converted features and input are input to the spatial attention module The initial features of are summed together to get the final output feature of the spatial attention module, which has a size of W×H×C.

The spatial attention module is composed of a convolutional layer, Batch Normalization, Relu activation function, matrix transposition and multiplication layer, softmax layer, and data transposition and addition layer. Through the spatial attention module, the network can find The region of semantic information, thereby improving the accuracy of hyperspectral image classification.

2c) Set up the spectral attention mechanism module, which consists of a matrix transpose and multiply layer, a softmax layer, and a data transpose and add layer, and its structure is shown in Figure 6.

In the matrix transposition and multiplication layer in the spectral attention module of the present invention, the input feature size is W×H×C, and the feature size is first converted to C×N, where N=W×H, and then the converted feature is performed Matrix transposition, the size of the obtained features is N×C, and then matrix multiplication is performed on the features obtained through feature conversion and matrix transposition to obtain an output matrix, whose size is C×C.

The softmax layer in the spectral attention module of the present invention uses softmax to normalize the matrix output from the matrix transposition and multiplication layer, normalizes the values in the matrix to between 0 and 1, and simultaneously normalizes The matrix of is multiplied by the features obtained through matrix transformation to obtain the output features, whose size is C×N.

In the data transposition and addition layer in the spectral attention module of the present invention, the output result of the softmax layer is converted first, and the size after conversion is W×H×C, and then the converted features are input to the spectral attention module The initial features of are summed together to get the final output features of the spectral attention module, which is of size W×H×C.

The spectral attention module is composed of a matrix transposition and multiplication layer, a softmax layer, and a data transposition and addition layer. Through the spectral attention module, the network can find spectral bands with strong spectral information, thereby improving the performance of hyperspectral image classification. accuracy and robustness of the network.

2d) Set the pixel-level attention mechanism module, which is composed of convolutional layer, Batch Normalization and Relu activation function connected in sequence, and its structure is shown in Figure 7.

In the convolution layer in the pixel-level attention module of the present invention, the size of the convolution kernel is set to 1*1, and the feature enhancement is performed on each pixel of the hyperspectral image through the convolution operation of the convolution kernel with a size of 1*1, thus Improved accuracy for hyperspectral image classification.

2e) Set the fully connected layer. The fully connected layer is composed of the first fully connected layer, the second fully connected layer and the softmax layer. The convolution kernel size of the first fully connected layer is 9800×1024, and the volume of the second fully connected layer The product kernel size is 1024×16. where 16 is the total number of categories in the input hyperspectral image.

The first fully connected layer in the fully connected layer of the present invention has a feature input size of 1×9800, first multiplies a coefficient for each value of the input feature, and then inputs it to the first fully connected layer, and its output feature size is 1× 1024. The multiplied coefficients are an initialized set of vectors conforming to the Gaussian distribution, and the coefficients can be updated during the sample training process.

The second fully connected layer in the fully connected layer of the present invention has a feature input size of 1×1024 and an output feature size of 1×16.

2f) Connect the Octave convolution module, the spatial attention module, the spectral attention module and the fully connected layer set above in sequence to obtain the Octave convolutional neural network.

Step 3 determines the loss function of the convolutional neural network:

(3a) Input the training sample set {T ₁ , T ₂ ,...,T _j ,...,T _M } into the Octave convolution module of the Octave convolutional neural network, and output the feature F of the last layer of the convolutional layer.

(3b) Input the last layer feature F into the spatial attention module and spectral attention module of the Octave convolutional neural network, the output features are A and B respectively, and then input the output features A and B to the pixel-level attention module , the output features are C and D respectively.

(3c) Input the obtained features C and D to the fully connected layer of the Octave convolutional neural network, and output the output classification results obtained by using the features C and D; meanwhile, multiply the features C and D by a coefficient, and then perform step by step The pixels are added to obtain the fused feature E, and then the feature E is input to the fully connected layer of the Octave convolutional neural network, and the output classification result obtained by using the fused feature E is output, and the loss function loss of the Octave convolutional neural network is obtained _op :

in: is the L2 norm of the Octave convolutional neural network weight vector, and η is hyperparameters; Indicates the cross entropy between the output classification result and the actual result after using the fused feature E through the fully connected layer, y _j is the predicted class label probability of T _j in the training image library, o _j is the actual class label of T _j in the training image library , the input of loss ₁ is the predicted class label probability obtained after the fused feature E passes through the fully connected layer.

The principle of loss ₂ and loss ₃ is the same as the formula expression and loss _1. Loss ₂ is the cross entropy between the output classification result and the actual result after the feature C passes through the fully connected layer. Loss ₃ is the output classification result and the actual result after the feature D passes through the fully connected layer. The cross entropy of the result; the input of loss ₂ is the predicted class label probability obtained after feature C passes through the fully connected layer, and the input of loss ₃ is the predicted class label probability obtained after feature D passes through the fully connected layer.

Step 4, perform iterative training on the convolutional neural network.

The existing methods for iterative training of the Octave convolutional neural network include the gradient descent optimization algorithm, the Nesterov gradient acceleration method, and the Adagrad method. In this example, but not limited to the gradient descent algorithm, the implementation steps are as follows:

4a) Set the number of iterations to P, set the initial learning rate of training to L, and the decay rate to β, and input the training image library {T ₁ ,T ₂ ,…,T _j ,…,T _M } points to the one constructed in step 2 In the convolutional neural network, the number of pictures input each time is Q, and the number of times is G:

Where M is the total number of samples in the training image library.

4b) Let the learning rate l corresponding to each input picture be:

l= ^L *βG

4c) Update the parameters of the Octave convolutional neural network G times by the following formula to obtain the updated weight vector W _new :

Among them, W is the weight vector of Octave convolutional neural network parameters.

Bring the updated weight vector W _new into the loss function loss _{op in 3c) to obtain the loss function loss op after} the weight vector is updated.

4d) Input the next training picture to the Octave convolutional neural network, and update the loss function loss _op after the weight vector is updated, so that the value of the loss function loss _op keeps decreasing.

4e) Repeat 4d) until the loss function loss _op no longer declines, and the current number of training rounds is less than the set number of iterations P, then stop the training of the network to obtain the trained Octave convolutional neural network; otherwise, when the training rounds When the set number of iterations P is reached, the training of the network is stopped, and the trained Octave convolutional neural network is obtained.

In this example, the gradient descent algorithm is used to optimize the network to find the optimal solution, but the network optimization is not limited to the gradient descent algorithm, and other optimization algorithms such as genetic algorithms can still optimize the network.

Step 5 classifies the test sample set.

Input the normalized test sample set into the trained Octave convolutional neural network, and obtain the classification result of the test sample set of the input hyperspectral image from the output of the trained Octave convolutional neural network, and complete the hyperspectral image classification. Accurate classification of images.

The invention mainly solves the problems in the prior art that the distance between the same category is large, the distance between different categories is small, and the classification accuracy is low. The present invention builds a training sample set and a test sample set by establishing a hyperspectral image library and corresponding categories of the image library, and randomly selecting a specified number of hyperspectral image samples from each type of hyperspectral image after normalization processing; constructing a training sample set and a test sample set; Octave convolution module, spatial attention module, spectral attention module, pixel-level attention module and Octave convolutional neural network of fully connected layer; input training samples from training sample set into Octave convolutional neural network to obtain training samples Classify the results, and determine the loss function of the convolutional neural network; iteratively update the loss function through the gradient descent method until the loss value is stable, and obtain the trained Octave convolutional neural network; the normalized test sample set to be classified Input to the trained Octave convolutional neural network to obtain classification results. The invention has high classification accuracy and strong robustness, and can be applied to the analysis and management of hyperspectral image data.

The effect of the invention can be further illustrated by the following simulation:

Example 8

The hyperspectral image classification method based on the deep learning of the spatial spectrum attention mechanism of Octave convolution is the same as that of Embodiment 1-7,

Simulation conditions

This example completes the present invention and the existing remote sensing image scene classification simulation on the HP-Z840-Workstation with Xeon(R) CPU E5-2630, GeForce TITAN XP, 64G RAM, Ubuntu system, and TensorFlow operating platform.

The simulation parameters are set as follows, the iteration round P is 175 times, the initial learning rate L is 0.00001, η=0.0001, the number of input pictures Q is 16 each time, and the decay rate β is 0.9. The training sequence is that in each iterative training, the class label discriminator and the classification difference optimizer are jointly trained.

Simulation content

Download the Indian Pines hyperspectral image dataset. The Indian Pines hyperspectral image dataset was imaged by the Airborne Visible Infrared Imaging Spectrometer (AVIRIS) in 1992 on an Indian pine tree in Indiana, USA. See Figure 3, Figure 3(a) is The original Indian Pines hyperspectral image, Figure 3(b) is the class label of the corresponding pixel of the original Indian Pines hyperspectral image. With each pixel as the center, select a sliding frame of size 13*13, and slide pixel by pixel. Each time you slide to get an image, all the images obtained by sliding are used to build a hyperspectral image library {I ₁ ,I ₂ , ...,I _n ,...,I _N }. Then normalize the established hyperspectral image library, that is, first obtain the maximum pixel value V′ _max and the minimum pixel value V′ _min of the hyperspectral image library, and then calculate the values of all pixels in the hyperspectral image library Divide by the difference between V′ _max and V′ _min to get the normalized hyperspectral image library.

A certain number of hyperspectral images are randomly selected from the normalized hyperspectral image library as the training sample set D _T , and the remaining hyperspectral images are used as the test sample set D _t .

There are 16 types of images in the training sample set and test sample set, namely Aflalfa, Corn-notill, Corn-mintill, Corn, Grass-pasture, Grass-trees, Grass-pasture-mowed, Hay-windrowed, Oats, Soybean-nottill, Soybean-mintill, Soybean-clean, Wheat, Woods, Stone-Steel-Towers, Buildings-Grass-Trees-Drives; see Table 1 for the number of training samples, the number of test samples, and the total number of samples for each category.

Table 1 Category statistics table of Indian Pines hyperspectral image dataset

Under the above-mentioned simulation conditions, the training sample set D _T is used to train respectively with the present invention and the existing representative three image classification models, and the test sample set D _T is used to test, and the classification accuracy is compared. The results are shown in Table 2 .

Table 2 The performance evaluation table of the present invention and the existing hyperspectral image classification model

test model test sample accuracy this invention 0.9898 KFRC-CKIR 0.9860 2-DCNN 0.9888 3D-SRNet 0.9720

In Table 2, KFRC-CKIR is the existing hyperspectral classification method based on kernel regularization and kernel fusion, 2-DCNN is the existing hyperspectral image classification method based on deep convolutional neural network, and 3D-SRNet is the existing hyperspectral image classification method based on three-dimensional separable and Transfer Learning for Hyperspectral Image Classification.

As can be seen from Table 2, the convolutional neural network trained by the present invention is used to classify the test sample set D _t , and its accuracy rate is the highest among all the classification methods participating in the test, which is higher than that of the existing representative hyperspectral image classification model. accuracy has been improved.

To sum up, the invention discloses a method for classifying hyperspectral images based on spatial spectral attention based on Octave convolution, which solves the problems in the prior art that the distance between the same category is large, the distance between different categories is small, and the classification accuracy is low. The plan is: image input to be classified and data preprocessing, division of training set and test set, construction of Octave convolutional neural network, determination of Octave convolutional neural network loss function, training update of Octave convolutional neural network, test set data testing, completed Hyperspectral Image Classification. The present invention uses Octave convolution operation to strengthen feature representation, introduces spatial attention mechanism and spectral attention mechanism, and enables the network to more accurately find regions that are more favorable for classification and contain more comprehensive and detailed information. The invention has high classification accuracy and strong robustness, and can be applied to the analysis and management of hyperspectral image data.

Claims (6)

1. a kind of hyperspectral image classification method of the empty spectrum attention mechanism deep learning based on Octave convolution, feature exist In comprising the following steps that

(1) image input and data prediction: high spectrum image to be sorted is inputted, is carried out centered on each pixel by picture Vegetarian refreshments sliding, all image blocks slided are for establishing high spectrum image library { I₁,I₂,…,I_n,…,I_N, in image library The corresponding classification of each image block is { Y₁,Y₂,…,Y_n,…,Y_N, and place is normalized to the high spectrum image library of foundation It manages, wherein I_nN-th image block, Y in representative image library_nThe corresponding classification of n-th image block in representative image library, n representative image N-th of sample number in library, n ∈ [0, N], N represent the image block total number in high spectrum image library；

(2) training set and test set are divided: selecting specified quantity at random from every class high spectrum image after normalized High spectrum image sample constructs training sample set { T₁,T₂,…,T_j,…,T_M, using remaining high spectrum image as test specimens This collection { t₁,t₂,…t_d,…,t_mWherein T_jIndicate j-th of sample in training sample, j ∈ [0, M], t_dIt indicates the in test sample D sample, d ∈ [0, m], M are the total number of training sample, and m is the total number of test sample, m < N, M < N；

(3) Octave convolutional neural networks are built: building an Octave convolutional neural networks, the input terminal of network is Octave Convolution module, the output end of network are the output of full articulamentum as a result, including two between the input terminal and output end of network Branch, wherein a branch successively passes through space transforms power module and Pixel-level pays attention to power module, another branch successively passes through Spectrum notices that power module and Pixel-level pay attention to power module；

(4) Octave convolutional neural networks loss function loss is determined_op: setting loss function includes that will mention after Fusion Features The feature taken is input to the cross entropy loss of full articulamentum output category result and actual result obtained from₁, will by sky Between notice that power module and Pixel-level notice that power module is extracted feature be input to full articulamentum output category result obtained from With the cross entropy loss of actual result₂, by by spectrum pay attention to power module and Pixel-level pay attention to power module extract feature input To the cross entropy loss of full articulamentum output category result and actual result obtained from₃With the convolutional Neural for having hyper parameter Four part of L2 norm of network weight W, the loss function of network is successively added by above four part to be constituted；

(5) training updates: the number of iterations that network training is arranged is P, by gradient decline optimization to Octave convolutional Neural net Network is iterated training, until loss function loss_opDo not decline or exercise wheel number reaches the number of iterations, obtains trained Octave convolutional neural networks；

(6) test sample collection after normalized data test: is input to trained Octave convolutional Neural net In network, classification results are obtained, complete image classification.

2. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution Class method, which is characterized in that Octave convolutional neural networks described in step (3) are built, Octave convolution module therein, Space transforms power module, spectrum notice that power module, Pixel-level notice that power module and full articulamentum, parameter setting are as follows:

Octave convolution module, that is, the input module is made of, each conventional part sequentially connected four conventional parts It again include Octave convolution, Batch Normalization and Relu activation primitive is gone back between second and third conventional part There is a maximum pond layer；

The spatial attention mechanism module is turned by convolutional layer, Batch Normalization, Relu activation primitive, matrix It sets with the layer that is multiplied, softmax layers and data transposition with layer is added and constitutes；

The spectrum attention mechanism module, by matrix transposition and the layer that is multiplied, softmax layers and data transposition be added layer structure At；

The Pixel-level attention mechanism module, by convolutional layer, Batch Normalization and Relu activation primitive structure At；

The full articulamentum is made of the first full articulamentum and the second full articulamentum and softmax layers, i.e. output layer.

3. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution Class method, which is characterized in that determination Octave convolutional neural networks loss function loss described in step (4)_op, specifically include Following steps:

(4a) is by training image library { T₁,T₂,…,T_j,…,T_MIt is input to the Octave convolution mould of Octave convolutional neural networks Block exports the last layer feature F of convolutional layer；

(4b) the last layer feature F is separately input to the space transforms power module of Octave convolutional neural networks and spectrum pays attention to Power module, output feature is respectively A and B, then output feature A and B are input to Pixel-level and pay attention to power module, output feature difference For C and D.

The feature C and D that (4c) will be obtained, are input to the full articulamentum of Octave convolutional neural networks, and output utilizes feature C and D Obtained output category result；Simultaneously then feature C and D are added pixel-by-pixel, are merged respectively multiplied by a coefficient Feature E afterwards, then feature E is input to the full articulamentum of Octave convolutional neural networks, output is obtained using fused feature E The output category result arrived obtains the loss function loss of Octave convolutional neural networks_op:

Wherein, loss₁For using fused feature E output category result and actual result after full articulamentum cross entropy, loss₂It is characterized the cross entropy of C output category result and actual result after full articulamentum, loss₃D is characterized by connecting entirely The cross entropy of output category result and actual result after layer is connect,For the L2 norm of convolutional neural networks weight vectors, η isHyper parameter.

4. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 3 based on Octave convolution Class method, which is characterized in that the output category result obtained using fused feature E and actual result in step (4c) Cross entropy loss₁, specific cross entropy formula is as follows:

Wherein, y_jFor T in training image library_jPrediction category probability, o_jFor T in training image library_jPractical category；loss₁'s Input is the prediction category probability that fused feature E is obtained after full articulamentum；

loss₂、loss₃Principle and formula and loss₁It is identical, loss₂Input be characterized C obtained after full articulamentum it is pre- Survey category probability, loss₃Input be characterized the prediction category probability that D is obtained after full articulamentum.

5. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution Class method, which is characterized in that step is normalized high spectrum image library in (1), is carried out by following formula:

Wherein V_maxFor the point maximum value of all pixels in high spectrum image library, V_minFor the point of all pixels in high spectrum image library Minimum value, V_nFor the pixel value at any point in high spectrum image library, { I '₁,I′₂,…,I′_n,…,I′_NFor after normalized High spectrum image library, I '_nFor n-th of sample of high spectrum image after normalized, n ∈ [0, N].

6. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution Class method, which is characterized in that training is iterated to convolutional neural networks by gradient decline optimization in step (5), is realized It is as follows:

The initial learning rate of (5a) setting training is L, attenuation rate β, by training image library { T₁,T₂,…,T_j,…,T_MIt is divided into G In the convolutional neural networks of secondary input building, the number of pictures inputted every time is Q, then:

Wherein M is the total number of training image library sample；

(5b) sets the corresponding learning rate l of input picture every time are as follows:

L=L* β^G

(5c) carries out the update of G subparameter to convolutional neural networks by following formula, obtains updated weight vectors W_new；

Wherein, W is the weight vectors of convolutional neural networks parameter；

(5d) will train picture to input convolutional neural networks, loss function loss updated to weight vectors next time_opIt carries out It updates, so that loss function loss_opValue constantly decline；

(5e) repeats (5d), until loss function loss_opNo longer decline, and current exercise wheel number is less than the number of iterations of setting P then stops the training to the network, obtains trained Octave convolutional neural networks；Otherwise, when training round reaches setting The number of iterations P when, stop training to the network, obtain trained Octave convolutional neural networks.