CN111639587B - Hyperspectral image classification method based on multi-scale spectrum space convolution neural network - Google Patents
Hyperspectral image classification method based on multi-scale spectrum space convolution neural network Download PDFInfo
- Publication number
- CN111639587B CN111639587B CN202010461596.0A CN202010461596A CN111639587B CN 111639587 B CN111639587 B CN 111639587B CN 202010461596 A CN202010461596 A CN 202010461596A CN 111639587 B CN111639587 B CN 111639587B
- Authority
- CN
- China
- Prior art keywords
- layer
- activation function
- convolution
- function layer
- scale
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
- G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Abstract
The invention discloses a hyperspectral image classification method of a multi-scale spectrum space-convolution neural network, which mainly solves the problems that in the prior art, only single-scale features are extracted during inter-spectrum feature extraction and space feature extraction, and the classification effect on ground object categories with non-concentrated sample distribution or small sample amount is poor. The implementation scheme is as follows: 1) Inputting a hyperspectral image, and generating a training sample set and a test sample set with different sample numbers; 2) Constructing a multi-scale spectrum space convolution neural network; 3) Inputting the training set into a multi-scale spectrum space convolution neural network to obtain a prediction category, calculating hinge cross entropy loss by using the prediction category and a real label, and training the network by using a random gradient descent method until the hinge cross entropy loss is converged; 4) And inputting the test sample into the trained multi-scale spectrum space convolution neural network to obtain a classification result. The method can obtain high-accuracy classification under the condition of few training samples, and can be used for detecting the ground object types of the hyperspectral images.
Description
Technical Field
The invention belongs to the technical field of remote sensing information processing, and further relates to a hyperspectral image classification method which can be used for detecting the types of ground objects of hyperspectral images.
Background
The hyperspectrum records the continuous spectrum characteristics of the ground object target by rich wave band information, and has the possibility of recognizing more types of ground object targets and classifying the ground objects with higher precision. The key of the hyperspectral image classification technology is to classify the sample categories by utilizing the spatial characteristics and the inter-spectral characteristics of the hyperspectral images. The method has important significance in the aspects of land resource assessment and disaster monitoring. However, the existing classification method mainly depends on a large number of training samples, and due to the fact that sample labels are difficult to obtain, the overfitting problem is easily caused under the condition of few samples, and the sample classification accuracy is further influenced.
A3-D Deep Learning frame (IEEE Transactions on science and Remote Sensing, 2017. The method takes an original three-dimensional cube as input data, in an end-to-end spectrum-space residual error network, an inter-spectrum residual block and a space residual block continuously learn identification characteristics from abundant spectrum and space information of a hyperspectral image, spectrum characteristics obtained by the inter-spectrum residual block and space characteristics obtained by the space residual block are fused in a cascading mode, and finally the fused high-level spectrum and space characteristics are input into a classification layer to classify the hyperspectral image. According to the method, a convolution operation of a single scale is adopted in the inter-spectrum and space residual block, and only the features of the single scale are extracted, so that the effect of the whole network on the classification of the hyperspectral image is poor.
The northwest industrial university discloses a hyperspectral image classification method based on space-spectrum combination of a deep convolutional neural network in an authorized patent document 'hyperspectral image classification method based on space-spectrum combination of a deep convolutional neural network' (authorization publication number: CN 105320965B). The method comprises the following specific steps: firstly, training a convolutional neural network by using a small amount of label data, and autonomously extracting the spatial spectral feature of a hyperspectral image by using the network without any compression and dimension reduction processing; then, training a Support Vector Machine (SVM) classifier by using the extracted space spectrum characteristics to classify the images; and finally, combining the trained deep convolutional neural network and the trained classifier to obtain a deep convolutional neural network structure DCNN-SVM which can autonomously extract and classify the spatial spectral features of the hyperspectral image. According to the method, the extraction of the hollow spectral features in the hyperspectral image by the deep convolutional neural network is still in a single scale, so that the effect of the SVM classifier on the hyperspectral image classification is poor.
Except the listed hyperspectral image classification methods, the existing hyperspectral image classification methods based on the deep convolutional neural network are similar to the two methods, and the common characteristic of the methods is that a feature extraction module adopts single-scale convolution operation, and multi-scale space and inter-spectrum features are not extracted on a feature level, so that the methods have poor hyperspectral image classification effect when few samples are trained.
Disclosure of Invention
The invention aims to provide a hyperspectral image classification method based on a multi-scale spectrum space convolution neural network aiming at the defects of the prior art, so as to improve the precision of ground object target classification in a hyperspectral image under the condition of less sample training.
The idea of achieving the purpose of the invention is to construct a multi-scale space spectrum characteristic extraction module and a multi-scale space characteristic extraction module, then generate a multi-scale space spectrum convolution neural network, input hyperspectral three-dimensional data into the multi-scale space spectrum characteristic module to extract multi-scale space spectrum characteristics, input the multi-scale space spectrum characteristics into the multi-scale space characteristic extraction module, extract multi-scale space spectrum combination characteristics and classify, train the network by using a hinge cross entropy loss function, and finally input a test sample into the trained multi-scale space spectrum convolution neural network to classify hyperspectral images. The hyperspectral image classification model can solve the problem of hyperspectral image classification under the condition of few samples.
In order to achieve the above purpose, the specific steps of the invention comprise:
1. a hyperspectral image classification method based on a multiscale spectral space convolution neural network is characterized by comprising the following steps:
(1) Inputting an original three-dimensional hyperspectral image, and performing 0 edge filling operation on the original three-dimensional hyperspectral image; selecting a hyperspectral image block by taking each pixel point as a center in the hyperspectral image after the filling operation;
(2) Generating training sample sets and testing sample sets with different sample numbers for the hyperspectral image blocks;
(3) Constructing a multi-scale spectrum space convolution neural network:
(3a) Constructing a multi-scale inter-spectrum feature extraction module formed by connecting two inter-spectrum residual modules in series;
(3b) Constructing a multi-scale spatial feature extraction module formed by connecting two spatial residual modules in series;
(3c) Sequentially connecting a multi-scale spectrum inter-characteristic extraction module, a multi-scale space characteristic extraction module and a softmax classifier in series to form a multi-scale spectrum space convolution neural network;
(4) Training a multi-scale spectrum space convolution neural network:
(4a) Constructing a hinge cross entropy loss function, wherein the formula is as follows:
wherein L represents the hinge intersection between the predicted tag vector and the true tag vectorThe cross entropy loss value, argmax (·) denotes the position where the maximum value is taken, y denotes the predictor tag vector, y * Representing the true tag vector, theta represents a set threshold,
representing the i-th element, y, in the true tag vector i Representing the ith element in the prediction label vector, and M represents the total number of classes in the training set;
(4b) Inputting the training sample set and the training sample labels into a multi-scale spectrum space convolution neural network to obtain a prediction label y of the training sample i (ii) a Training the multi-scale spectrum space-convolution neural network by adopting a random gradient descent algorithm until the hinge cross entropy loss function is converged to obtain the trained multi-scale spectrum space-convolution neural network;
(5) And inputting the test sample into the trained multi-scale spectrum space convolution neural network to obtain a class label of the test sample, and completing the classification of the hyperspectral image.
Compared with the prior art, the invention has the following advantages:
firstly, because the invention constructs the multi-scale spatial feature extraction block and the multi-scale inter-spectrum feature extraction block, and extracts the multi-scale inter-spectrum spatial features with higher resolution and marking in the hyperspectral image, the invention overcomes the defect that only single scale features are extracted during inter-spectrum feature extraction and spatial feature extraction in the prior art when few samples are trained, and improves the classification precision of ground objects in the hyperspectral image.
Secondly, because the novel hinge cross entropy loss function is constructed, the multi-scale spectrum space-convolution neural network can pay more attention to samples which are difficult to classify, the defect that the loss function in the prior art has poor classifying effect on ground object types with non-concentrated sample distribution or small sample amount is overcome, and the classifying capability of the multi-scale spectrum space-convolution neural network during training of few samples is improved.
Drawings
FIG. 1 is a flow chart of an implementation of the present invention;
FIG. 2 is a schematic diagram of a model structure of the multi-scale spectral space-convolution neural network in the present invention.
FIG. 3 is a schematic diagram of the structure of the inter-spectral residual block in the multi-scale spectral space-convolution neural network of the present invention.
FIG. 4 is a simulation diagram of classification results of the present invention and the prior art on two different data sets.
Detailed Description
Embodiments and effects of the present invention will be further described below with reference to the accompanying drawings.
Referring to fig. 1, implementation steps for the present example include the following.
Step 1, inputting a hyperspectral image.
The hyperspectral image is three-dimensional data S epsilon R a×b×c Each wave band in the hyperspectral image corresponds to a two-dimensional matrix S in the three-dimensional data i ∈R a×b Wherein, epsilon represents belonging to a symbol, R represents a real number field symbol, a represents the length of the hyperspectral image, b represents the width of the hyperspectral image, c represents the spectrum wave band number of the hyperspectral image, i represents the serial number of the spectrum wave band in the hyperspectral image, i =1,2, \ 8230;, c.
And 2, acquiring a hyperspectral image block set.
And performing 0 edge filling operation on the original three-dimensional hyperspectral image data, wherein the size of 0 pixel of the edge filling is k. In the filled hyperspectral image, each pixel point is taken as the center, and an image block with the space size of (2k + 1) × (2k + 1) and the channel number d is selected to obtain a hyperspectral image block set, wherein the channel number d is the same as the spectral band number of the hyperspectral image, and the example is but not limited to k =4.
And 3, generating a training sample set and a testing sample set with different sample numbers from the hyperspectral image block set.
Firstly, distributing a hyperspectral image block into a set to which a central pixel point type belongs according to the type of the central pixel point;
then, selecting image blocks in each category set according to the proportion of 0.01 as a training set, and taking a central pixel point label of each image block in the training set as a label of the image block;
and finally, taking the residual image blocks in each category set as a test set.
This example takes a 0.01 ratio for each class of targets as the training set, leaving 0.99 as the test set for two different hyperspectral datasets, kennedy Space Center and Salinas scene.
And 4, constructing a multi-scale spectrum space convolution neural network.
Referring to fig. 2, the specific implementation steps of the present embodiment include the following.
4.1 To build a multi-scale inter-spectrum feature extraction module composed of two inter-spectrum residual modules connected in series, wherein:
the inter-spectrum residual error module structure sequentially comprises: the first convolution layer → the first activation function layer → the first mean layer → the second convolution layer → the second activation function layer → the first fusion layer → the third convolution layer → the third activation function layer → the second fusion layer → the fourth convolution layer → the fourth activation function layer → the first carboxylate splice layer → the fifth convolution layer → the fifth activation function layer → the second mean layer → the sixth convolution layer → the sixth activation function layer → the third fusion layer → the seventh convolution layer → the seventh activation function layer → the fourth fusion layer → the eighth convolution layer → the eighth activation function layer → the second carboxylate splice layer; the structure is shown in fig. 3.
The first uniform layer is used for uniformly dividing the feature graph output by the first activation function layer into four parts;
the first fusion layer is used for adding the second activation function layer and a third part of the first activation function layer;
the second fusion layer is used for adding the third activation function layer and the fourth part of the first activation function layer;
the first splice layer is used for splicing the first parts of the second activation function layer, the third activation function layer, the fourth activation function layer and the first activation function layer together;
the second uniform layer is used for uniformly dividing the feature map output by the fifth activation function layer into four parts;
the third fusion layer is used for adding the sixth activation function layer and a third part of the fifth activation function layer;
the fourth fusion layer is used for adding the seventh activation function layer and a fourth part of the fifth activation function layer;
the second splice layer is used for splicing the first parts of the sixth, seventh, eighth and fifth activation function layers together;
the parameters of each layer are set as follows:
the convolution kernels of all three-dimensional convolution layers in the inter-spectrum residual module are set to be 1 × 7, and the number of the convolution kernels is set to be 15; setting the activation function of each activation function layer in the inter-spectrum residual error module as a ReLU activation function;
4.2 Building a multi-scale spatial feature extraction module consisting of two spatial residual modules connected in series, wherein:
the structure of the space residual error module is as follows in sequence: the 1 st convolution layer → the 1 st activation function layer → the 1 st mean layer → the 2 nd convolution layer → the 2 nd activation function layer → the 1 st fusion layer → the 3 rd convolution layer → the 3 rd activation function layer → the 2 nd fusion layer → the 4 th convolution layer → the 4 th activation function layer → the 1 st linkage splice layer → the 5 th convolution layer → the 5 th activation function layer → the 2 nd mean layer → the 6 th convolution layer → the 6 th activation function layer → the 3 rd fusion layer → the 7 th convolution layer → the 7 th activation function layer → the 4 th fusion layer → the 8 th convolution layer → the 8 th activation function layer → the 2 nd linkage splice layer; the structure is the same as that of fig. 3.
The 1 st uniform layer is used for uniformly dividing the feature map output by the 1 st activation function layer into four parts;
the 1 st fusion layer is used for adding the 2 nd activation function layer and a third copy of the 1 st activation function layer;
the 2 nd fusion layer is used for adding the 3 rd activation function layer and the fourth part of the 1 st activation function layer;
the 1 st splice layer is used for splicing the 2 nd, 3 rd, 4 th and 1 st activation function layers together;
the 2 nd uniform layer is used for uniformly dividing the feature map output by the 5 th activation function layer into four parts;
the 3 rd fusion layer is used for adding the 6 th activation function layer and a third copy of the 5 th activation function layer;
the 4 th fusion layer is used for adding the 7 th activation function layer and the fourth part of the 5 th activation function layer;
the 1 st splice layer is used for splicing the first parts of the 6 th, 7 th, 8 th and 5 th activation function layers together;
the parameters of each layer are as follows:
the convolution kernel size of all convolution layers in the space residual module is set to be 3 x 128, and the number of the convolution kernels is set to be 15;
and setting the activation function of each activation function layer in the spatial residual module as a ReLU activation function.
4.3 A multi-scale spectrum inter-feature extraction module, a multi-scale space feature extraction module and a softmax classifier are sequentially connected in series to form a multi-scale spectrum space convolution neural network.
And 5, constructing a hinge cross entropy loss function.
In the hyperspectral image classification task, as some ground object target samples are not distributed in a concentrated manner or the amount of the target samples is small, common cross entropy loss functions cannot well distinguish the difficult samples, in order to enable a network to pay more attention to the difficult samples, the embodiment designs a novel hinge cross entropy loss function, and the formula is as follows:
wherein L represents the hinge cross entropy loss value between the predicted tag vector and the real tag vector, argmax (·) represents the position of the maximum value, y represents the predicted tag vector, y represents the hinge cross entropy loss value * Representing the true tag vector, theta represents a set threshold,
representing the i-th element, y, in the true tag vector i Representing the ith element in the predictive label vector and M representing the total number of classes in the training set.
And 6, training the multi-scale spectrum space convolution neural network.
Inputting a training sample set and a training sample label into a multi-scale spectrum space convolution neural network;
training the multi-scale spectrum space convolution neural network by adopting a random gradient descent algorithm to obtain a prediction label y of the training sample, wherein the prediction label y is the same as the real label y * And (5) calculating the hinge cross entropy loss through the formula in the step (5) until the hinge cross entropy loss is converged to obtain the trained multi-scale spectrum space convolution neural network.
And 7, classifying the test sample set.
And inputting the test sample into the trained multi-scale spectrum space convolution neural network to obtain the category of the test sample, and finishing the classification of the hyperspectral image.
The effect of the present invention will be further explained with the simulation experiment.
1. Simulation experiment conditions are as follows:
the hardware platform of the simulation experiment of the invention is as follows: intercore i7-6700, frequency 3.4GHz, nvidia GeForce GTX1080Ti. The software for the simulation experiment of the present invention uses a pytorch.
The simulation experiment of the invention is to classify the ground object targets in Kennedy Space Center and Salinas scene hyperspectral data sets respectively by adopting the invention and two existing 3D-Densenet and SSRN methods.
The 3D-Densenet method refers to: chunjn Zhang et al, in "Three-dimensional dense connected volumetric network for hyperspectral Remote Sensing image classification" (Journal of Applied Remote Sensing, 2019), proposed a method for classifying hyperspectral images using a 3D dense connection network, referred to as the 3D-DenseNet method for short.
The SSRN classification method comprises the following steps: a method for classifying Hyperspectral images by using an end-to-end spectrum-space Residual error Network is proposed in Spectral-Spatial information Network for Hyperspectral Image Classification (IEEE Transactions on Geoscience and Remote Sensing, 2017.
The Kennedy Space Center and Salinas scene hyperspectral datasets used in the present invention are data collected by the AVIRIS sensor at Kennedy Space Center and Salinas Valley, calif., respectively. The data set used in the simulation experiment of the invention is collected from a website:
http:// www.ehu.eus/ccwittco/index.phitile = Hyperspectral _ Remote _ Sensing _ Scenes. The size of the image of the Kennedy Space Center hyperspectral dataset is 512 x 614, the image has 176 bands and comprises 13 types of ground objects, and the type and the number of each type of ground object are shown in Table 1.
TABLE 1 Kennedy Space Center sample classes and quantities
The size of the Salinas scene hyperspectral dataset image is 512 x 217, the Salinas scene hyperspectral dataset image has 204 wave bands and comprises 16 types of ground objects, and the category and the number of each type of ground object are shown in Table 2.
TABLE 2 Salinas scene sample classes and quantities
| Class label | Class of ground object | Number of |
| 1 | Brocoli_green_weeds_1 | 2009 |
| 2 | Brocoli_green_weeds_2 | 3726 |
| 3 | Fallow | 1976 |
| 4 | Fallow_rough_plow | 1394 |
| 5 | Fallow_smooth | 2678 |
| 6 | Stubble | 3959 |
| 7 | Celery | 3579 |
| 8 | Graps_untrained | 11271 |
| 9 | Soil_vinyard_develop | 6203 |
| 10 | Corn_senesced_green_weeds | 3278 |
| 11 | Lettuce_romaine_4wk | 1068 |
| 12 | Lettuce_romaine_5wk | 1927 |
| 13 | Lettuce_romaine_6wk | 916 |
| 14 | Lettuce_romaine_7wk | 1070 |
| 15 | Vinyard_untrained | 7268 |
| 16 | Vinyard_vertical_trellis | 1807 |
2. Simulation experiment content and result analysis:
2.1 For verifying the high efficiency and good classification performance of the invention, three evaluation indexes of overall classification accuracy OA, average accuracy AA and Kappa coefficient are adopted.
The overall classification accuracy OA refers to the proportion of the number of correctly classified pixels on the test set divided by the total number of pixels, and the value is between 0% and 100%, and the larger the value is, the better the classification effect is.
The average precision AA is that the number of correctly classified pixel points of each type on the test set is divided by the total number of all pixels of the type to obtain the correct classification precision of the type, and the precision of all the types is averaged to obtain a value between 0 and 100 percent, wherein the larger the value is, the better the classification effect is.
The Kappa coefficient is an evaluation index defined on the confusion matrix, elements on the diagonal line of the confusion matrix and elements deviating from the diagonal line are comprehensively considered, the classification performance of the algorithm is objectively reflected, the value of the Kappa coefficient is between-1 and 1, and the larger the value is, the better the classification effect is.
The classification accuracy of the Kennedy Space Center and Salinas scene hyperspectral datasets of the present invention and two prior art techniques were compared and the results are shown in Table 3.
TABLE 3 comparison of classification accuracy of three networks under different data sets
As can be seen from Table 3, the method of the present invention obtains a higher classification accuracy under the data sets of Kennedy Space Center and Salinas scene, which indicates that the method of the present invention can more accurately predict the category of the hyperspectral image sample.
2.2 The results of simulation classification on two different datasets, kennedy Space Center and Salinas scene, using the method of the present invention and the two prior art techniques described above, are shown in fig. 4. Wherein:
FIG. 4a is a graph showing the results of classification on a Kennedy Space Center hyperspectral dataset using a prior art 3D-DenseNet;
FIG. 4b is a graph showing the results of classification using the existing SSRN on a Kennedy Space Center hyperspectral dataset;
FIG. 4c is a graph showing the results of classification on the Kennedy Space Center hyperspectral dataset using the method of the present invention;
FIG. 4D is a graph showing the results of classification using the existing 3D-DenseNet on the Salinas scene hyperspectral dataset;
FIG. 4e is a graph showing the results of classification using the existing SSRN on the Salinas scene hyperspectral dataset;
FIG. 4f is a diagram showing the classification result on the Salinas scene hyperspectral dataset by the method of the present invention;
as can be seen from FIG. 4c, the classification results of the present invention on the Kennedy Space Center dataset are clearly smoother and more edge-defined than those of FIGS. 4a,4 b.
As can be seen from FIG. 4f, the classification result graph of the Salinas scene data set of the invention is obviously smoother than that of FIGS. 4d and 4e, the edges are clearer, and the regional target consistency is better. This directly illustrates that the method proposed by the present invention can generate a higher quality hyperspectral image classification result map than the two existing techniques.
The above simulation experiments show that: the method can acquire multi-scale spectrum space characteristics by using the constructed multi-scale space and inter-spectrum characteristic extraction module, can acquire a hyperspectral image classification result by building a multi-scale spectrum space convolution neural network, and trains the neural network by adopting a hinge cross entropy loss function, so that the multi-scale spectrum space convolution neural network focuses more on ground object categories with unconcentrated sample distribution or small sample amount. The method solves the problem that the classification accuracy is low under the condition of few training samples due to the fact that single-scale convolution operation is adopted in spectral and spatial features in the prior art, and is a very practical hyperspectral image classification method under the condition of few training samples.
Claims (7)
1. A hyperspectral image classification method based on a multiscale spectral space convolution neural network is characterized by comprising the following steps:
(1) Inputting an original three-dimensional hyperspectral image, and performing 0 edge filling operation on the original three-dimensional hyperspectral image; selecting a hyperspectral image block by taking each pixel point as a center in the hyperspectral image after the filling operation;
(2) Generating training sample sets and testing sample sets with different sample numbers for the hyperspectral image blocks;
(3) Constructing a multi-scale spectrum space convolution neural network:
(3a) Constructing a multi-scale inter-spectrum feature extraction module formed by connecting two inter-spectrum residual modules in series;
(3b) Constructing a multi-scale spatial feature extraction module formed by connecting two spatial residual modules in series;
(3c) Sequentially connecting a multi-scale spectrum inter-feature extraction module, a multi-scale spatial feature extraction module and a softmax classifier in series to form a multi-scale spectrum space convolution neural network;
(4) Training a multi-scale spectrum space convolution neural network:
(4a) Constructing a hinge cross entropy loss function, wherein the formula is as follows:
wherein L represents the hinge cross entropy loss value between the predicted tag vector and the real tag vector, argmax (·) represents the position of the maximum value, y represents the predicted tag vector, y represents the hinge cross entropy loss value * Representing the true tag vector, theta represents a set threshold,
representing the i-th element, y, in the true tag vector i Representing the ith element in the prediction label vector, and M represents the total number of classes in the training set;
(4b) Inputting the training sample set and the training sample label into a multi-scale spectrum space convolution neural network to obtain a prediction label y of the training sample i (ii) a Training the multi-scale spectrum space-convolution neural network by adopting a random gradient descent algorithm until the hinge cross entropy loss function is converged to obtain the trained multi-scale spectrum space-convolution neural network;
(5) And inputting the test sample into the trained multi-scale spectrum space convolution neural network to obtain a class label of the test sample, and completing the classification of the hyperspectral image.
2. The method of claim 1, wherein the original three-dimensional hyperspectral image in (1) is a three-dimensional data Sec R a×b×c Each wave band in the hyperspectral image corresponds to three-dimensional dataA two-dimensional matrix S of i ∈R a×b Wherein, epsilon represents belonging to a symbol, R represents a real number field symbol, a represents the length of the hyperspectral image, b represents the width of the hyperspectral image, c represents the spectrum wave band number of the hyperspectral image, i represents the serial number of the spectrum wave band in the hyperspectral image, i =1,2, \ 8230;, c.
3. The method of claim 1, wherein the 0 edge fill operation in (1) is to fill the hyperspectral image edge with a size k of 0 pixels.
4. The method according to claim 1, wherein the spatial size of the hyperspectral image block in (1) is (2k + 1) x (2k + 1), the number of channels d is the same as the number of spectral bands of the hyperspectral image, and k is the size of 0 pixel filled in the edge of the hyperspectral image.
5. The method according to claim 1, wherein in (2), training sample sets and test sample sets with different sample numbers are generated, and the hyperspectral image blocks are firstly distributed into sets to which the categories belong according to the categories of central pixel points of the hyperspectral image blocks; selecting image blocks in each class of set according to the proportion of 0.01 as a training set, and taking the central pixel point label of each image block as the label of the image block; and respectively using the residual image blocks in each class set as test sets.
6. The method of claim 1, wherein the structure of each multi-scale inter-spectrum residual module in (3 a) is, in order: the first convolution layer → the first activation function layer → the first mean layer → the second convolution layer → the second activation function layer → the first fusion layer → the third convolution layer → the third activation function layer → the second fusion layer → the fourth convolution layer → the fourth activation function layer → the first carboxylate splice layer → the fifth convolution layer → the fifth activation function layer → the second mean layer → the sixth convolution layer → the sixth activation function layer → the third fusion layer → the seventh convolution layer → the seventh activation function layer → the fourth fusion layer → the eighth convolution layer → the eighth activation function layer → the second carboxylate splice layer;
the first uniform layer is used for uniformly dividing the feature graph output by the first activation function layer into four parts;
the first fusion layer is used for adding the second activation function layer and a third part of the first activation function layer;
the second fusion layer is used for adding the third activation function layer and the fourth part of the first activation function layer;
the first splice layer is used for splicing the first parts of the second activation function layer, the third activation function layer, the fourth activation function layer and the first activation function layer together;
the second uniform layer is used for uniformly dividing the feature map output by the fifth activation function layer into four parts;
the third fusion layer is used for adding the sixth activation function layer and a third part of the fifth activation function layer;
the fourth fusion layer is used for adding the seventh activation function layer and a fourth part of the fifth activation function layer;
the second splice layer is used for splicing the first parts of the sixth, seventh, eighth and fifth activation function layers together;
the convolution kernels of all convolution layers in the multi-scale inter-spectrum module are set to be 1 × 7, and the number of the convolution kernels is set to be 15;
and setting the activation function of each activation function layer in the multi-scale inter-spectrum module as a ReLU activation function.
7. The method of claim 1, wherein the structure of each multi-scale space module in (3 b) is, in turn: the 1 st convolution layer → the 1 st activation function layer → the 1 st mean layer → the 2 nd convolution layer → the 2 nd activation function layer → the 1 st fusion layer → the 3 rd convolution layer → the 3 rd activation function layer → the 2 nd fusion layer → the 4 th convolution layer → the 4 th activation function layer → the 1 st linkage splice layer → the 5 th convolution layer → the 5 th activation function layer → the 2 nd mean layer → the 6 th convolution layer → the 6 th activation function layer → the 3 rd fusion layer → the 7 th convolution layer → the 7 th activation function layer → the 4 th fusion layer → the 8 th convolution layer → the 8 th activation function layer → the 2 nd linkage splice layer;
the 1 st uniform layer is used for uniformly dividing the feature map output by the 1 st activation function layer into four parts;
the 1 st fusion layer is used for adding the 2 nd activation function layer and a third copy of the 1 st activation function layer;
the 2 nd fusion layer is used for adding the 3 rd activation function layer and the fourth part of the 1 st activation function layer;
the 1 st splice layer is used for splicing the 2 nd, 3 rd, 4 th and 1 st activation function layers together;
the 2 nd uniform layer is used for uniformly dividing the feature map output by the 5 th activation function layer into four parts;
the 3 rd fusion layer is used for adding the 6 th activation function layer and a third part of the 5 th activation function layer;
the 4 th fusion layer is used for adding the 7 th activation function layer and the fourth part of the 5 th activation function layer;
the 1 st splice layer is used for splicing the first parts of the 6 th, 7 th, 8 th and 5 th activation function layers together;
the convolution kernel size of all convolution layers in the multi-scale space module is set to be 3 x 128, and the number of the convolution kernels is set to be 15;
and setting the activation function of each activation function layer in the multi-scale space module as a ReLU activation function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010461596.0A CN111639587B (en) | 2020-05-27 | 2020-05-27 | Hyperspectral image classification method based on multi-scale spectrum space convolution neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010461596.0A CN111639587B (en) | 2020-05-27 | 2020-05-27 | Hyperspectral image classification method based on multi-scale spectrum space convolution neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111639587A CN111639587A (en) | 2020-09-08 |
| CN111639587B true CN111639587B (en) | 2023-03-24 |
Family
ID=72331129
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010461596.0A Active CN111639587B (en) | 2020-05-27 | 2020-05-27 | Hyperspectral image classification method based on multi-scale spectrum space convolution neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111639587B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112668421B (en) * | 2020-12-18 | 2022-04-29 | 武汉大学 | Attention mechanism-based rapid classification method for hyperspectral crops of unmanned aerial vehicle |
| CN113095145B (en) * | 2021-03-15 | 2022-08-16 | 南京理工大学 | Hyperspectral anomaly detection deep learning method based on pixel pair matching and double-window discrimination |
| CN112949592B (en) * | 2021-03-31 | 2022-07-22 | 云南大学 | Hyperspectral image classification method and device and electronic equipment |
| CN113095409B (en) * | 2021-04-13 | 2023-04-07 | 西安电子科技大学 | Hyperspectral image classification method based on attention mechanism and weight sharing |
| CN115187870B (en) * | 2022-09-13 | 2023-01-03 | 浙江蓝景科技有限公司杭州分公司 | Marine plastic waste material identification method and system, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107316013A (en) * | 2017-06-14 | 2017-11-03 | 西安电子科技大学 | Hyperspectral image classification method with DCNN is converted based on NSCT |
| WO2017215284A1 (en) * | 2016-06-14 | 2017-12-21 | 山东大学 | Gastrointestinal tumor microscopic hyper-spectral image processing method based on convolutional neural network |
| WO2018045626A1 (en) * | 2016-09-07 | 2018-03-15 | 深圳大学 | Super-pixel level information fusion-based hyperspectral image classification method and system |
| CN110084159A (en) * | 2019-04-15 | 2019-08-02 | 西安电子科技大学 | Hyperspectral image classification method based on the multistage empty spectrum information CNN of joint |
| CN110321963A (en) * | 2019-07-09 | 2019-10-11 | 西安电子科技大学 | Based on the hyperspectral image classification method for merging multiple dimensioned multidimensional sky spectrum signature |
| CN111191736A (en) * | 2020-01-05 | 2020-05-22 | 西安电子科技大学 | Hyperspectral image classification method based on depth feature cross fusion |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106157307B (en) * | 2016-06-27 | 2018-09-11 | 浙江工商大学 | A kind of monocular image depth estimation method based on multiple dimensioned CNN and continuous CRF |
| US10846566B2 (en) * | 2016-09-14 | 2020-11-24 | Konica Minolta Laboratory U.S.A., Inc. | Method and system for multi-scale cell image segmentation using multiple parallel convolutional neural networks |
| CN108985238B (en) * | 2018-07-23 | 2021-10-22 | 武汉大学 | Impervious surface extraction method and system combining deep learning and semantic probability |
-
2020
- 2020-05-27 CN CN202010461596.0A patent/CN111639587B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017215284A1 (en) * | 2016-06-14 | 2017-12-21 | 山东大学 | Gastrointestinal tumor microscopic hyper-spectral image processing method based on convolutional neural network |
| WO2018045626A1 (en) * | 2016-09-07 | 2018-03-15 | 深圳大学 | Super-pixel level information fusion-based hyperspectral image classification method and system |
| CN107316013A (en) * | 2017-06-14 | 2017-11-03 | 西安电子科技大学 | Hyperspectral image classification method with DCNN is converted based on NSCT |
| CN110084159A (en) * | 2019-04-15 | 2019-08-02 | 西安电子科技大学 | Hyperspectral image classification method based on the multistage empty spectrum information CNN of joint |
| CN110321963A (en) * | 2019-07-09 | 2019-10-11 | 西安电子科技大学 | Based on the hyperspectral image classification method for merging multiple dimensioned multidimensional sky spectrum signature |
| CN111191736A (en) * | 2020-01-05 | 2020-05-22 | 西安电子科技大学 | Hyperspectral image classification method based on depth feature cross fusion |
Non-Patent Citations (2)
| Title |
|---|
| SSCDenseNet:一种空-谱卷积稠密网络的高光谱图像分类算法;刘启超等;《电子学报》;20200415(第04期);全文 * |
| 基于卷积神经网络与主动学习的高光谱图像分类;宋晗等;《中国科学院大学学报》;20200315(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111639587A (en) | 2020-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111639587B (en) | Hyperspectral image classification method based on multi-scale spectrum space convolution neural network | |
| CN110321963B (en) | Hyperspectral image classification method based on fusion of multi-scale and multi-dimensional space spectrum features | |
| CN111274865B (en) | Remote sensing image cloud detection method and device based on full convolution neural network | |
| Li et al. | Adaptive multiscale deep fusion residual network for remote sensing image classification | |
| CN110969088B (en) | Remote sensing image change detection method based on significance detection and deep twin neural network | |
| CN102646200B (en) | Image classifying method and system for self-adaption weight fusion of multiple classifiers | |
| CN111080629A (en) | Method for detecting image splicing tampering | |
| CN103440505B (en) | The Classification of hyperspectral remote sensing image method of space neighborhood information weighting | |
| CN110263717A (en) | It is a kind of incorporate streetscape image land used status determine method | |
| CN106295124A (en) | Utilize the method that multiple image detecting technique comprehensively analyzes gene polyadenylation signal figure likelihood probability amount | |
| CN104299232B (en) | SAR image segmentation method based on self-adaptive window directionlet domain and improved FCM | |
| CN112052755A (en) | Semantic convolution hyperspectral image classification method based on multi-path attention mechanism | |
| CN109766936A (en) | Image change detection method based on information transmitting and attention mechanism | |
| CN105427309A (en) | Multiscale hierarchical processing method for extracting object-oriented high-spatial resolution remote sensing information | |
| CN110414616B (en) | Remote sensing image dictionary learning and classifying method utilizing spatial relationship | |
| CN108229551A (en) | A kind of Classification of hyperspectral remote sensing image method based on compact dictionary rarefaction representation | |
| CN106485238A (en) | A kind of high-spectrum remote sensing feature extraction and sorting technique and its system | |
| Ge et al. | Deep residual network-based fusion framework for hyperspectral and LiDAR data | |
| CN105512622A (en) | Visible remote-sensing image sea-land segmentation method based on image segmentation and supervised learning | |
| CN115222994A (en) | Hyperspectral image classification method based on hybrid spectrum network and multi-head self-attention mechanism | |
| CN106529472A (en) | Target detection method and apparatus based on large-scale high-resolution and high-spectral image | |
| CN112052758A (en) | Hyperspectral image classification method based on attention mechanism and recurrent neural network | |
| CN114332534B (en) | Hyperspectral image small sample classification method | |
| CN109635726B (en) | Landslide identification method based on combination of symmetric deep network and multi-scale pooling | |
| CN111242028A (en) | Remote sensing image ground object segmentation method based on U-Net |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |