CN111695467A - Spatial spectrum full convolution hyperspectral image classification method based on superpixel sample expansion - Google Patents
Spatial spectrum full convolution hyperspectral image classification method based on superpixel sample expansion Download PDFInfo
- Publication number
- CN111695467A CN111695467A CN202010485713.7A CN202010485713A CN111695467A CN 111695467 A CN111695467 A CN 111695467A CN 202010485713 A CN202010485713 A CN 202010485713A CN 111695467 A CN111695467 A CN 111695467A
- Authority
- CN
- China
- Prior art keywords
- convolution
- spatial
- spectral
- hyperspectral image
- hyperspectral
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
- G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Abstract
The invention discloses a spatial spectrum full convolution hyperspectral image classification method based on superpixel sample expansion, which comprises the steps of inputting a hyperspectral image; acquiring a training set and a test set; performing principal component analysis and dimensionality reduction on the hyperspectral image; carrying out entropy rate segmentation on the dimensionality reduction result; generating a pseudo label sample; updating the training set; performing data preprocessing on the hyperspectral image; inputting a convolutional neural network; training a convolutional neural network, and classifying the hyperspectral images; repeating the operations and voting; and outputting a hyperspectral classification result. According to the method, the pseudo label samples are expanded by utilizing the entropy rate superpixel segmentation result, the prior characteristics of the hyperspectral images are fully utilized, the number of samples is increased, the problems of network overfitting and low network convergence speed are solved, and the accuracy of hyperspectral image classification under the condition of rare marked samples is improved.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a spatial spectrum full convolution hyperspectral image classification method based on superpixel sample expansion.
Background
With the progress of scientific technology, the hyperspectral remote sensing technology is greatly developed. The hyperspectral data can be represented as a hyperspectral data cube, which is a three-dimensional data structure. The hyperspectral data can be regarded as a three-dimensional image, and one-dimensional spectral information is added in addition to a common two-dimensional image. The space image describes two-dimensional space characteristics of the earth surface, and the spectrum dimension of the space image reveals the spectrum curve characteristics of each pixel of the image, so that the organic fusion of the space dimension and the spectrum dimension information of the remote sensing data is realized. The hyperspectral remote sensing image contains abundant spectral information, can provide spatial domain information and spectral domain information, has the characteristic of 'map integration', can realize accurate identification and detail extraction of ground objects, and provides favorable conditions for knowing an objective world. Due to the unique characteristics of hyperspectral images, the hyperspectral remote sensing technology is widely applied to different fields. In the civil field, hyperspectral remote sensing images have been used in the aspects of urban environment monitoring, surface soil monitoring, geological exploration, disaster assessment, agricultural yield estimation, crop analysis and the like. The hyperspectral remote sensing technology is widely applied to daily life of people. Therefore, designing a practical and efficient hyperspectral image classification method has become an indispensable scientific and technological requirement of modern society.
At present, researchers have proposed many classical classification methods for hyperspectral image classification, and representative classification methods are Support Vector Machines (SVMs) and neural networks (CNNs). The SVM obtains a better classification result in small sample classification by maximizing the class boundary. The SVM is introduced into the hyperspectral image Classification in the Classification of hyperspectral remote sensing images with supported vector machines by F.Melgani and L.Bruzzone, the best Classification result is obtained at that time, but the SVM determines a kernel function, the experience judgment is completely needed, and the Classification performance is poor due to the selection of an improper kernel function. Meanwhile, with the rise of deep learning, the convolutional neural network is also applied to the field of hyperspectral image classification. However, because a large number of labeled samples are needed for training the convolutional neural network as training samples, and the labeling cost of the hyperspectral image is very expensive, how to solve the problem of small samples is a popular direction at present.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a method for classifying a hyperspectral image based on a spatial-spectral full-convolution of a superpixel sample expansion, so as to improve the classification effect of the hyperspectral image by using a segmentation result as prior information.
The invention adopts the following technical scheme:
the method for classifying the spatial spectrum full convolution hyperspectral image based on the super-pixel sample expansion comprises the following steps:
s1, inputting the hyperspectral image PaviaU, and acquiring the training sample X from the hyperspectral image PaviaUtAnd test sample Xe；
S2, carrying out normalization processing on the hyperspectral data set, and adding n labels which are in the neighborhood of each training sample from the segmentation label matrix and are the same as the segmentation labels of the training samples into the training samples as pseudo label samples;
s3, respectively constructing a spectral feature extraction module and a spatial spectrum feature extraction module, constructing a spectral feature map and a spatial spectrum feature map weighted fusion module, taking the features of spatial spectrum combination as input, passing through two convolution layers and setting, and constructing a spatial spectrum combined full convolution neural network for hyperspectral classification;
s4, constructing a loss function of the step S3 full convolution neural network, and training the neural network;
and S5, obtaining a final classification result graph through multiple training votes, and realizing image classification.
Specifically, step S1 specifically includes:
s101, recording a three-dimensional hyperspectral image PaviaU as
U, V and C are the space length, space width and spectrum channel number of the hyperspectral image respectively, the hyperspectral image comprises N pixel points, each pixel point has C spectrum wave bands, and N is U × V;
s102, randomly taking 30 samples of class labels 1 to 9 in X to form an initial training sample set XtThe rest is used as a test sample Xe。
Specifically, step S2 specifically includes:
s201, carrying out PCA (principal component analysis) dimensionality reduction on the three-dimensional hyperspectral image, wherein the number of channels of the image after dimensionality reduction is 1;
s202, carrying out entropy rate superpixel segmentation on the image subjected to PCA dimensionality reduction to obtain 50 segmentation results, and obtaining a segmentation label matrix of
S203, setting a real label matrix
Partition the label matrix into
(x0,y0) The real label of the training sample is
The segmentation labels in the segmentation map are
7 × 7 with (x, y) as center
Generating a pseudo label of
And expanding the pseudo label samples meeting the standard, wherein the number of the training samples is n +1 times of the original number, and the test samples are kept unchanged.
Specifically, step S3 specifically includes:
s301, constructing a spectral feature extraction module, wherein the spectral feature extraction module comprises three convolution layers and a merging layer, and a relu activation function and batch normalization processing are added behind each convolution layer;
s302, constructing a spatial spectrum feature extraction module, wherein the spatial spectrum feature extraction module comprises a 1 × 1 convolution layer, a relu activation layer, a batch normalization layer, a multi-scale spatial feature fusion layer, a 3 × 3 cavity convolution layer, a relu activation layer, a batch normalization layer, a 2 × 2 average pooling layer and a merging layer;
s303, constructing a spectral feature map and spatial spectrum feature map weighting fusion module;
s304, the characteristics of the combination of the space spectrum are used as input to pass through the two convolution layers;
s305, carrying out PCA dimension reduction on the feature map after convolution to 5 dimensions for use in subsequent CRF treatment;
and S306, performing Softmax operation on the convolved feature map to output a classification probability matrix, and outputting the dimension number with the largest value in the classification probability matrix as a prediction class label to obtain a classification result.
Further, in step S301, the batch normalization processing parameters are: and (3) momentum is 0.8, the sizes of convolution kernels are all 1, the step length is 1, the number of channels after all convolution is 64, and the convolution results are added after three continuous convolutions to obtain a spectral feature map.
Further, in step S302, the first convolutional layer uses 1 × 1 convolution with a step size of 1; the contrast rate of the 3 × 3 hole convolution is 2, and the step size is 1; the number of all convolution result channels is 64; all batch normalization processing parameters were: the momentum is 0.8, the merging layer is the sum of the feature maps of the three convolution layers, and the number of channels is held at 64.
Further, in step S303, the spectral feature map and the spatial feature map are weighted and superimposed as follows:
Cunite＝λspectralCspectral+λspatialCspatial
wherein, CuniteFor the weighted feature map, λspectralAnd λspatialWeighting coefficients, C, for trainable spectral and spatial features in the network, respectivelyspectralAnd CspatialRespectively a spectral feature map and a spatial feature map.
Specifically, in step S4, the loss function is:
L＝L1+L2
where L is the final loss function, L1And L2Respectively labeled samples and pseudo-labeled samples in the training set,
and
the label representing the ith training sample and the prediction label, and taking 1 or 2 for j represents that the sample is an original sample or a pseudo label sample.
Specifically, step S5 specifically includes:
s501, adding the feature graph after the PCA dimensionality reduction to 5 dimensionalities in the step S3 and the classification result obtained by inputting the normalized hyperspectral data in the step S4 into a network into a conditional random field;
s502, obtaining a classification result of primary training through a conditional random field;
s503, repeating the pseudo sample expansion and network training operations for m times on the same training sample to obtain m classification results, and outputting the prediction class label with the largest occurrence frequency of each pixel as a final prediction class label.
Further, in step S501, the energy function of the conditional random field is as follows:
wherein psiu(yi) And psip(yi,yj) Respectively, a unary function part and a binary function part.
Compared with the prior art, the invention has at least the following beneficial effects:
according to the method for classifying the empty-spectrum full-convolution hyperspectral images based on the super-pixel sample expansion, the result generated by the hyperspectral image segmentation is used for guiding the generation of the pseudo label sample, and the prior information of the hyperspectral images is effectively used for expanding the training sample, so that the training sample can still keep good classification accuracy under the condition of small samples; the method adopts a space-spectrum combination mode to extract the features, so that the spectrum and space-spectrum features of the hyperspectral image can be more fully extracted, and the classification accuracy of the hyperspectral image is improved; the spatial feature extraction module uses the void convolution of different resolution rates to realize multi-scale feature fusion, and spatial features of the hyperspectral image are extracted on multiple scales; a voter is added before the final classification result, so that the robustness of the whole structure is enhanced, and the classification result is more stable and reliable; the full convolution neural network is adopted, a full connection layer is not introduced, and end-to-end image classification can be realized on hyperspectral images of any size; the input of the full convolution neural network is preprocessed hyperspectral data with the same size as the original hyperspectral image, and the condition of high redundancy of training data caused by the fact that Patch of each pixel is used as input data in the conventional method is avoided.
Furthermore, the entropy rate superpixel is set to obtain the segmentation label, the prior information of the hyperspectral image is effectively utilized, a sample similar to a training sample is supplemented into a training set under the condition that a classification label is not needed, and the difficulty that the hyperspectral image can be used for training marked samples is effectively solved.
Furthermore, the full convolution neural network used for constructing the full convolution neural network combined by the empty spectrum for hyperspectral classification has no full connection layer, so that a hyperspectral image with any size can be very conveniently accepted as input. The spatial-spectral combination mode can combine spectral information and spatial information in a hyperspectral image into a new feature with better effect relative to a single spatial feature or a spectral feature.
Further, the feature extraction of the spectrum module uses continuous 1 × 1 convolutional layers to complete the extraction of the spectrum features on the premise that the influence on the spatial information is as small as possible, and the existence of the residual module enables the gradient information to be preserved so that the model can be converged better.
Furthermore, the feature extraction of the spatial spectrum module uses the hole convolution of different contrast rates, so that the receptive field can be expanded and multi-scale spatial information can be extracted at the same time.
Furthermore, aiming at the proposed pseudo label sample expansion method, the loss function of the full convolution neural network is constructed to make corresponding improvement, and the cross entropy of the pseudo label sample is also added into the loss function, so that the network can be converged better.
Furthermore, aiming at the instability of the proposed pseudo label sample expansion method caused by self limitation, the final classification diagram obtained by training and voting for multiple times effectively increases the robustness of the model.
In conclusion, the spatial spectrum full-convolution hyperspectral image classification method based on superpixel sample expansion effectively utilizes the prior information of the hyperspectral image to realize pseudo sample expansion, solves the problem that the hyperspectral image has marked samples, and simultaneously fully utilizes multi-scale spatial features and spectral features to realize higher classification precision by the full-convolution hyperspectral network of the spatial spectrum.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
FIG. 1 is a flow chart of an implementation of the present invention;
FIG. 2 is a block diagram of a flow chart of an implementation of pseudo tag exemplar augmentation of the present invention;
FIG. 3 is a multi-scale spatial feature fusion module in accordance with the present invention.
Detailed Description
The invention provides a spatial spectrum full convolution hyperspectral image classification method based on superpixel sample expansion, which is used for inputting a hyperspectral image; acquiring a training set and a test set; performing principal component analysis and dimensionality reduction on the hyperspectral image; performing entropy rate superpixel segmentation on the dimensionality reduction result; generating a pseudo label sample; updating the training set; performing data preprocessing on the hyperspectral image; inputting a full convolution neural network; training a full convolution neural network, and classifying the hyperspectral images; repeating the operations and voting; and outputting a hyperspectral classification result. According to the method, the pseudo label samples are expanded by utilizing the entropy rate superpixel segmentation result, the spatial prior information of the hyperspectral images is fully utilized, the number of samples is increased, the problem of network overfitting is solved, and the accuracy, the classification efficiency and the classification performance of the hyperspectral images under the condition of small samples are effectively improved.
Referring to fig. 1, the present invention provides a method for classifying a spatial spectrum full convolution hyperspectral image based on superpixel sample expansion, which comprises the following steps:
s1, inputting a hyperspectral image PaviaU, wherein the hyperspectral three-dimensional image is data used in the experiment, and acquiring a training sample X from the PaviaU hyperspectral imagetAnd test sample Xe；
S101, recording a three-dimensional hyperspectral image PaviaU as
The hyperspectral image comprises N pixel points, each pixel point is provided with C spectral bands, N is equal to U × V.PaviaU data set, N is equal to 207400 samples, U is equal to 610, V is equal to 340, C is equal to 103, the class labels are 1 to 9, the samples are 42776 in total, X is normalized, and the data value is kept in [0, 1]The method comprises the following specific steps:
s102, randomly taking 30 samples with class labels of 1 to 9 in X to form an initial training sample set XtThe rest is used as a test sample Xe。
S2, after the hyperspectral data set is normalized, generating a segmentation label by utilizing the entropy rate superpixel to generate a pseudo label sample;
s201, carrying out PCA (principal component analysis) dimensionality reduction on the three-dimensional hyperspectral image, wherein the number of channels of the image after dimensionality reduction is 1;
s202, carrying out entropy rate superpixel segmentation on the image subjected to PCA dimensionality reduction to obtain 50 segmentation results, and obtaining a segmentation label matrix of
S203, setting a real label matrix
Partition the label matrix into
(x0,y0) The real label of the training sample is
The segmentation labels in the segmentation map are
7 × 7 with (x, y) as center
For which a pseudo label is generated as
The pseudo label samples meeting the above criteria are expanded, at this time, the number of training samples is changed to n +1 times of the original number, and the test samples are kept unchanged, as shown in fig. 2.
S3, constructing a full convolution neural network for hyperspectral classification and space spectrum combination;
s301, the spectral feature extraction module is composed of three convolution layers, and a relu activation function and batch normalization processing are added behind each convolution layer.
The batch normalization processing parameters were:
and (3) momentum is 0.8, the sizes of convolution kernels are all 1, the step length is 1, the number of channels after all convolution is 64, and the convolution results are added after three continuous convolutions to obtain a spectral feature map.
S302, the spatial spectrum feature extraction module consists of three convolution layers, the convolution kernel size of the first convolution layer is 1, and the step length is 1; the second convolution layer is used to realize multi-scale feature fusion, and has a structure shown in fig. 3, and is obtained by convolution and addition of three holes with convolution kernel size of 3, convolution rate of 2,3,4, and step size of 1; the third convolutional layer was followed by a 2 x 2 average pooling layer.
After each convolution is finished, performing relu activation function and batch normalization processing; the convolution kernel sizes of the third convolution layers are all 3, the distortion rate is 2, and the step size is 1.
All batch normalization processing parameters were:
and (4) momentum is 0.8, the number of channels of all convolution results is 64, and the convolution results are added after three times of continuous convolution to obtain a space spectrum feature map.
S303, weighting and superposing the spectral feature map and the spatial feature map as follows:
Cunite＝λspectralCspectral+λspatialCspatial
wherein, CuniteThe weighted feature map has a channel of 64 λspectralAnd λspatialWeighting coefficients, C, for trainable spectral and spatial features in the network, respectivelyspectralAnd CspatialRespectively a spectrum characteristic diagram and a space spectrum characteristic diagram;
s304, taking the characteristics of the combination of the space spectrum as input, passing through two 1 × 1 convolution layers, performing relu activation after convolution, wherein the sizes of convolution kernels are all 1, the step lengths are all 1, the number of channels of a first convolution result is 64, and the number of channels of a second convolution result is 128;
s305, carrying out PCA dimension reduction on the feature map after convolution to 5 dimensions for use in subsequent CRF treatment;
s306, performing Softmax operation on the convolved feature map to output a classification probability matrix of 610 multiplied by 340 multiplied by 9, and outputting the dimension number with the largest value in the 9 dimensions as a prediction class label to obtain a classification result with the size of 610 multiplied by 340.
S4, constructing a loss function of the full convolution neural network, and training the neural network;
s401, calculating the cross entropy of the training sample prediction label and the training sample label after the expansion by using the cross entropy through a loss function, wherein the cross entropy is shown as the following formula:
L＝L1+L2
where L is the final loss function, L1And L2Respectively labeled samples and pseudo-labeled samples in the training set,
and
a label and a prediction label representing the ith training sample, wherein j takes 1 or 2 to represent that the sample is an original sample or a pseudo label sample;
s402, inputting the normalized hyperspectral data into a network, and iterating 1000 times to generate a predicted label graph.
And S5, obtaining a final classification result graph through multiple training votes.
S501, adding the outputs of the step S305 and the step S402 into a conditional random field, wherein the energy function of the conditional random field is as follows,
ψu(yi) And psip(yi,yj) Are respectively provided withAre a unary function part and a binary function part.
In the present invention, the formula for calculating the unary part is ψu(yi)＝-log P(yi) Wherein, P (y)i) Is the label assignment probability of pixel i given by the proposed full convolution network.
The binary function part is defined as:
wherein if yi＝yj,μ(yi,yj) 1, otherwise zero; k is a radical ofmIs a gaussian kernel; f. ofiAnd fjIs the feature vector of pixels i and j in arbitrary feature space; omegamIs the corresponding weight; in order to fully utilize the deep spectral space characteristics, C in S305 isuniteThe first five principal components of (a) are used as features of each pixel. The complete form of the gaussian kernel is written as:
s502, obtaining a classification result of primary training through a conditional random field;
and S503, repeating the above m times of operations on the same training sample to obtain m times of classification results, and outputting the prediction class label with the maximum occurrence frequency of each pixel as a final prediction class label.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The three evaluation indexes of the simulation experiment are as follows:
the overall accuracy OA represents the proportion of correctly classified samples to all samples, with a larger value indicating a better classification. The average precision AA represents the average value of the classification precision of each class, and the larger the value is, the better the classification effect is. The chi-square coefficient Kappa represents different weights in the confusion matrix, and the larger the value is, the better the classification effect is.
The prior art contrast classification method used in the invention is as follows:
song et al, in "Hyperspectral Image Classification With Deep feature fusion Network, IEEE trans. Geosci. remote Sens., vol.56, No.6, pp.3173-3184, June 2018", propose a Hyperspectral Image Classification method, depth feature fusion DFFN method for short.
Table 1 is a classification result quantitative analysis table of the present invention (PaviaU data set is selected, 30 labeled samples of each class are used as training set:
in conclusion, the spatial spectrum full-convolution hyperspectral image classification method based on superpixel sample expansion effectively utilizes the prior information of the hyperspectral image to realize pseudo sample expansion, solves the problem that the hyperspectral image has marked samples, and simultaneously utilizes the spatial features and the spectral features of multiple scales to realize higher classification precision by the full-convolution hyperspectral network of the spatial spectrum.
The above-mentioned contents are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and any modification made on the basis of the technical idea of the present invention falls within the protection scope of the claims of the present invention.
Claims (10)
1. The method for classifying the spatial spectrum full convolution hyperspectral image based on the super-pixel sample expansion is characterized by comprising the following steps of:
s1, inputting the hyperspectral image PaviaU, and acquiring the training sample X from the hyperspectral image PaviaUtAnd test sample Xe；
S2, carrying out normalization processing on the hyperspectral data set, and adding n labels which are in the neighborhood of each training sample from the segmentation label matrix and are the same as the segmentation labels of the training samples into the training samples as pseudo label samples;
s3, respectively constructing a spectral feature extraction module and a spatial spectrum feature extraction module, constructing a spectral feature map and a spatial spectrum feature map weighted fusion module, taking the features of spatial spectrum combination as input, passing through two convolution layers and setting, and constructing a spatial spectrum combined full convolution neural network for hyperspectral classification;
s4, constructing a loss function of the step S3 full convolution neural network, and training the neural network;
and S5, obtaining a final classification result graph through multiple training votes, and realizing image classification.
2. The method for classifying the spatial-spectral full-convolution hyperspectral image based on the super-pixel sample expansion according to claim 1, wherein the step S1 is specifically as follows:
s101, recording a three-dimensional hyperspectral image PaviaU as
U, V and C are the space length, space width and spectrum channel number of the hyperspectral image respectively, the hyperspectral image comprises N pixel points, each pixel point has C spectrum wave bands, and N is U × V;
s102, randomly taking 30 samples of class labels 1 to 9 in X to form an initial training sample set XtThe rest is used as a test sample Xe。
3. The method for classifying the spatial-spectral full-convolution hyperspectral image based on the super-pixel sample expansion according to claim 1, wherein the step S2 is specifically as follows:
s201, carrying out PCA (principal component analysis) dimensionality reduction on the three-dimensional hyperspectral image, wherein the number of channels of the image after dimensionality reduction is 1;
s202, carrying out entropy rate superpixel segmentation on the image subjected to PCA dimensionality reduction to obtain 50 segmentation results, and obtaining a segmentation label matrix of
S203, setting a real label matrix
Partition the label matrix into
(x0,y0) The real label of the training sample is
The segmentation labels in the segmentation map are
7 × 7 with (x, y) as center
Generating a pseudo label of
And expanding the pseudo label samples meeting the standard, wherein the number of the training samples is n +1 times of the original number, and the test samples are kept unchanged.
4. The method for classifying the spatial-spectral full-convolution hyperspectral image based on the super-pixel sample expansion according to claim 1, wherein the step S3 is specifically as follows:
s301, constructing a spectral feature extraction module, wherein the spectral feature extraction module comprises three convolution layers and a merging layer, and a relu activation function and batch normalization processing are added behind each convolution layer;
s302, constructing a spatial spectrum feature extraction module, wherein the spatial spectrum feature extraction module comprises a 1 × 1 convolution layer, a relu activation layer, a batch normalization layer, a multi-scale spatial feature fusion layer, a 3 × 3 cavity convolution layer, a relu activation layer, a batch normalization layer, a 2 × 2 average pooling layer and a merging layer;
s303, constructing a spectral feature map and spatial spectrum feature map weighting fusion module;
s304, the characteristics of the combination of the space spectrum are used as input to pass through the two convolution layers;
s305, carrying out PCA dimension reduction on the feature map after convolution to 5 dimensions for use in subsequent CRF treatment;
and S306, performing Softmax operation on the convolved feature map to output a classification probability matrix, and outputting the dimension number with the largest value in the classification probability matrix as a prediction class label to obtain a classification result.
5. The method for classifying the spatial spectrum full-convolution hyperspectral image based on the super-pixel sample expansion according to claim 4, wherein in the step S301, batch normalization processing parameters are as follows: and (3) momentum is 0.8, the sizes of convolution kernels are all 1, the step length is 1, the number of channels after all convolution is 64, and the convolution results are added after three continuous convolutions to obtain a spectral feature map.
6. The method for classifying the spatial-spectral full-convolution hyperspectral image based on super-pixel sample expansion according to claim 4, wherein in the step S302, the convolution layer of the first layer uses 1 x 1 convolution with a step size of 1; the dispativity of the 3 × 3 hole convolution is 2, and the step size is 1; the number of all convolution result channels is 64; all batch normalization processing parameters were: the momentum is 0.8, the merging layer is the sum of the feature maps of the three convolution layers, and the number of channels is held at 64.
7. The method for classifying the spatial-spectral full-convolution hyperspectral images based on the super-pixel sample expansion according to claim 4, wherein in the step S303, the spectral feature map and the spatial-spectral feature map are weighted and superimposed as follows:
Cunite＝λspectralCspectral+λspatialCspatial
wherein, CuniteFor the weighted feature map, λspectralAnd λspatialWeighting coefficients, C, for trainable spectral and spatial features in the network, respectivelyspectralAnd CspatialRespectively a spectral feature map and a spatial feature map.
8. The method for classifying the spatial-spectral full-convolution hyperspectral image based on super-pixel sample expansion according to claim 1, wherein in step S4, the loss function is:
L＝L1+L2
where L is the final loss function, L1And L2Respectively labeled and pseudo-labeled in the training set, yj (i)And
the label representing the ith training sample and the prediction label, and taking 1 or 2 for j represents that the sample is an original sample or a pseudo label sample.
9. The method for classifying the spatial-spectral full-convolution hyperspectral image based on the super-pixel sample expansion according to claim 1, wherein the step S5 is specifically as follows:
s501, adding the feature graph after the PCA dimensionality reduction to 5 dimensionalities in the step S3 and the classification result obtained by inputting the normalized hyperspectral data in the step S4 into a network into a conditional random field;
s502, obtaining a classification result of primary training through a conditional random field;
s503, repeating the pseudo sample expansion and network training operations for m times on the same training sample to obtain m classification results, and outputting the prediction class label with the largest occurrence frequency of each pixel as a final prediction class label.
10. The method for classifying the spatial-spectral full-convolution hyperspectral image based on super-pixel sample expansion according to claim 9, wherein in step S501, an energy function of the conditional random field is as follows:
wherein psiu(yi) And psip(yi,yj) Respectively, a unary function part and a binary function part.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010485713.7A CN111695467B (en) | 2020-06-01 | 2020-06-01 | Spatial spectrum full convolution hyperspectral image classification method based on super-pixel sample expansion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010485713.7A CN111695467B (en) | 2020-06-01 | 2020-06-01 | Spatial spectrum full convolution hyperspectral image classification method based on super-pixel sample expansion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111695467A true CN111695467A (en) | 2020-09-22 |
| CN111695467B CN111695467B (en) | 2023-05-30 |
Family
ID=72479042
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010485713.7A Active CN111695467B (en) | 2020-06-01 | 2020-06-01 | Spatial spectrum full convolution hyperspectral image classification method based on super-pixel sample expansion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111695467B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112232137A (en) * | 2020-09-24 | 2021-01-15 | 北京航空航天大学 | Hyperspectral image processing method and device |
| CN112699756A (en) * | 2020-12-24 | 2021-04-23 | 中国农业科学院农业信息研究所 | Hyperspectral image-based tea origin identification method and system |
| CN112733769A (en) * | 2021-01-18 | 2021-04-30 | 西安电子科技大学 | Hyperspectral image classification method based on multiband entropy rate superpixel segmentation |
| CN112949592A (en) * | 2021-03-31 | 2021-06-11 | 云南大学 | Hyperspectral image classification method and device and electronic equipment |
| CN113052014A (en) * | 2021-03-09 | 2021-06-29 | 西北工业大学深圳研究院 | Hyperspectral image classification method based on double-layer space manifold representation |
| CN113222867A (en) * | 2021-04-16 | 2021-08-06 | 山东师范大学 | Image data enhancement method and system based on multi-template image |
| CN113239755A (en) * | 2021-04-28 | 2021-08-10 | 湖南大学 | Medical hyperspectral image classification method based on space-spectrum fusion deep learning |
| CN113327231A (en) * | 2021-05-28 | 2021-08-31 | 北京理工大学重庆创新中心 | Hyperspectral abnormal target detection method and system based on space-spectrum combination |
| CN113516194A (en) * | 2021-07-20 | 2021-10-19 | 海南长光卫星信息技术有限公司 | Hyperspectral remote sensing image semi-supervised classification method, device, equipment and storage medium |
| CN113642655A (en) * | 2021-08-18 | 2021-11-12 | 杭州电子科技大学 | Small sample image classification method based on support vector machine and convolutional neural network |
| CN113723255A (en) * | 2021-08-24 | 2021-11-30 | 中国地质大学（武汉） | Hyperspectral image classification method and storage medium |
| CN113822209A (en) * | 2021-09-27 | 2021-12-21 | 海南长光卫星信息技术有限公司 | Hyperspectral image recognition method and device, electronic equipment and readable storage medium |
| CN114049567A (en) * | 2021-11-22 | 2022-02-15 | 齐鲁工业大学 | Self-adaptive soft label generation method and application in hyperspectral image classification |
| CN114332534A (en) * | 2021-12-29 | 2022-04-12 | 山东省科学院海洋仪器仪表研究所 | Hyperspectral image small sample classification method |
| WO2023000160A1 (en) * | 2021-07-20 | 2023-01-26 | 海南长光卫星信息技术有限公司 | Hyperspectral remote sensing image semi-supervised classification method, apparatus, and device, and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017215284A1 (en) * | 2016-06-14 | 2017-12-21 | 山东大学 | Gastrointestinal tumor microscopic hyper-spectral image processing method based on convolutional neural network |
| CN109948693A (en) * | 2019-03-18 | 2019-06-28 | 西安电子科技大学 | Expand and generate confrontation network hyperspectral image classification method based on super-pixel sample |
| CN110321963A (en) * | 2019-07-09 | 2019-10-11 | 西安电子科技大学 | Based on the hyperspectral image classification method for merging multiple dimensioned multidimensional sky spectrum signature |
-
2020
- 2020-06-01 CN CN202010485713.7A patent/CN111695467B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017215284A1 (en) * | 2016-06-14 | 2017-12-21 | 山东大学 | Gastrointestinal tumor microscopic hyper-spectral image processing method based on convolutional neural network |
| CN109948693A (en) * | 2019-03-18 | 2019-06-28 | 西安电子科技大学 | Expand and generate confrontation network hyperspectral image classification method based on super-pixel sample |
| CN110321963A (en) * | 2019-07-09 | 2019-10-11 | 西安电子科技大学 | Based on the hyperspectral image classification method for merging multiple dimensioned multidimensional sky spectrum signature |
Non-Patent Citations (2)
| Title |
|---|
| 刘小波等: "深度迁移学习在高光谱遥感图像分类中的研究现状与展望", 《青岛科技大学学报(自然科学版)》 * |
| 张号逵等: "深度学习在高光谱图像分类领域的研究现状与展望", 《自动化学报》 * |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112232137A (en) * | 2020-09-24 | 2021-01-15 | 北京航空航天大学 | Hyperspectral image processing method and device |
| CN112699756A (en) * | 2020-12-24 | 2021-04-23 | 中国农业科学院农业信息研究所 | Hyperspectral image-based tea origin identification method and system |
| CN112699756B (en) * | 2020-12-24 | 2023-08-25 | 中国农业科学院农业信息研究所 | Hyperspectral image-based tea origin identification method and system |
| CN112733769A (en) * | 2021-01-18 | 2021-04-30 | 西安电子科技大学 | Hyperspectral image classification method based on multiband entropy rate superpixel segmentation |
| CN112733769B (en) * | 2021-01-18 | 2023-04-07 | 西安电子科技大学 | Hyperspectral image classification method based on multiband entropy rate superpixel segmentation |
| CN113052014A (en) * | 2021-03-09 | 2021-06-29 | 西北工业大学深圳研究院 | Hyperspectral image classification method based on double-layer space manifold representation |
| CN113052014B (en) * | 2021-03-09 | 2022-12-23 | 西北工业大学深圳研究院 | Hyperspectral image classification method based on double-layer space manifold representation |
| CN112949592A (en) * | 2021-03-31 | 2021-06-11 | 云南大学 | Hyperspectral image classification method and device and electronic equipment |
| CN113222867B (en) * | 2021-04-16 | 2022-05-20 | 山东师范大学 | Image data enhancement method and system based on multi-template image |
| CN113222867A (en) * | 2021-04-16 | 2021-08-06 | 山东师范大学 | Image data enhancement method and system based on multi-template image |
| CN113239755A (en) * | 2021-04-28 | 2021-08-10 | 湖南大学 | Medical hyperspectral image classification method based on space-spectrum fusion deep learning |
| CN113239755B (en) * | 2021-04-28 | 2022-06-21 | 湖南大学 | Medical hyperspectral image classification method based on space-spectrum fusion deep learning |
| CN113327231A (en) * | 2021-05-28 | 2021-08-31 | 北京理工大学重庆创新中心 | Hyperspectral abnormal target detection method and system based on space-spectrum combination |
| WO2023000160A1 (en) * | 2021-07-20 | 2023-01-26 | 海南长光卫星信息技术有限公司 | Hyperspectral remote sensing image semi-supervised classification method, apparatus, and device, and storage medium |
| CN113516194A (en) * | 2021-07-20 | 2021-10-19 | 海南长光卫星信息技术有限公司 | Hyperspectral remote sensing image semi-supervised classification method, device, equipment and storage medium |
| CN113516194B (en) * | 2021-07-20 | 2023-08-08 | 海南长光卫星信息技术有限公司 | Semi-supervised classification method, device, equipment and storage medium for hyperspectral remote sensing images |
| CN113642655A (en) * | 2021-08-18 | 2021-11-12 | 杭州电子科技大学 | Small sample image classification method based on support vector machine and convolutional neural network |
| CN113642655B (en) * | 2021-08-18 | 2024-02-13 | 杭州电子科技大学 | Small sample image classification method based on support vector machine and convolutional neural network |
| CN113723255A (en) * | 2021-08-24 | 2021-11-30 | 中国地质大学（武汉） | Hyperspectral image classification method and storage medium |
| CN113723255B (en) * | 2021-08-24 | 2023-09-01 | 中国地质大学（武汉） | Hyperspectral image classification method and storage medium |
| CN113822209A (en) * | 2021-09-27 | 2021-12-21 | 海南长光卫星信息技术有限公司 | Hyperspectral image recognition method and device, electronic equipment and readable storage medium |
| CN113822209B (en) * | 2021-09-27 | 2023-11-14 | 海南长光卫星信息技术有限公司 | Hyperspectral image recognition method and device, electronic equipment and readable storage medium |
| CN114049567A (en) * | 2021-11-22 | 2022-02-15 | 齐鲁工业大学 | Self-adaptive soft label generation method and application in hyperspectral image classification |
| CN114049567B (en) * | 2021-11-22 | 2024-02-23 | 齐鲁工业大学 | Adaptive soft label generation method and application in hyperspectral image classification |
| CN114332534A (en) * | 2021-12-29 | 2022-04-12 | 山东省科学院海洋仪器仪表研究所 | Hyperspectral image small sample classification method |
| CN114332534B (en) * | 2021-12-29 | 2024-03-29 | 山东省科学院海洋仪器仪表研究所 | Hyperspectral image small sample classification method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111695467B (en) | 2023-05-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111695467A (en) | Spatial spectrum full convolution hyperspectral image classification method based on superpixel sample expansion | |
| CN113011499B (en) | Hyperspectral remote sensing image classification method based on double-attention machine system | |
| CN109376804B (en) | Hyperspectral remote sensing image classification method based on attention mechanism and convolutional neural network | |
| CN110555446B (en) | Remote sensing image scene classification method based on multi-scale depth feature fusion and migration learning | |
| CN110443143B (en) | Multi-branch convolutional neural network fused remote sensing image scene classification method | |
| Othman et al. | Domain adaptation network for cross-scene classification | |
| Ham et al. | Investigation of the random forest framework for classification of hyperspectral data | |
| CN112347888B (en) | Remote sensing image scene classification method based on bi-directional feature iterative fusion | |
| CN110084159A (en) | Hyperspectral image classification method based on the multistage empty spectrum information CNN of joint | |
| He et al. | A dual global–local attention network for hyperspectral band selection | |
| CN113361485B (en) | Hyperspectral image classification method based on spectrum space attention fusion and deformable convolution residual error network | |
| CN110929080B (en) | Optical remote sensing image retrieval method based on attention and generation countermeasure network | |
| CN112347970B (en) | Remote sensing image ground object identification method based on graph convolution neural network | |
| CN115457311B (en) | Hyperspectral remote sensing image band selection method based on self-expression transfer learning | |
| CN113705580A (en) | Hyperspectral image classification method based on deep migration learning | |
| CN115909052A (en) | Hyperspectral remote sensing image classification method based on hybrid convolutional neural network | |
| CN110689065A (en) | Hyperspectral image classification method based on flat mixed convolution neural network | |
| CN115222998B (en) | Image classification method | |
| CN110188827A (en) | A kind of scene recognition method based on convolutional neural networks and recurrence autocoder model | |
| CN113642445A (en) | Hyperspectral image classification method based on full convolution neural network | |
| CN114419381A (en) | Semantic segmentation method and road ponding detection method and device applying same | |
| CN111626267A (en) | Hyperspectral remote sensing image classification method using void convolution | |
| CN111428758A (en) | Improved remote sensing image scene classification method based on unsupervised characterization learning | |
| CN116977747B (en) | Small sample hyperspectral classification method based on multipath multi-scale feature twin network | |
| CN116188981A (en) | Hyperspectral high-spatial-resolution remote sensing image classification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |