CN114048810A - Hyperspectral image classification method based on multilevel feature extraction network - Google Patents

Hyperspectral image classification method based on multilevel feature extraction network Download PDF

Info

Publication number
CN114048810A
CN114048810A CN202111326280.1A CN202111326280A CN114048810A CN 114048810 A CN114048810 A CN 114048810A CN 202111326280 A CN202111326280 A CN 202111326280A CN 114048810 A CN114048810 A CN 114048810A
Authority
CN
China
Prior art keywords
feature
feature extraction
hyperspectral
extraction network
hyperspectral image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111326280.1A
Other languages
Chinese (zh)
Inventor
陆小辰
杨德政
贾逢德
阳云龙
翟梦琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Donghua University
Original Assignee
Donghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Donghua University filed Critical Donghua University
Priority to CN202111326280.1A priority Critical patent/CN114048810A/en
Publication of CN114048810A publication Critical patent/CN114048810A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a hyperspectral image classification method based on a multilevel feature extraction network, which is characterized in that a multilevel joint feature extraction network is constructed, the multilevel joint feature extraction network acquires the most distinctive features on each wave band of a hyperspectral pixel by a lightweight framework, and converts the most distinctive features into global attention details for further joint feature extraction, so as to obtain deep semantic features and further improve the classification effect of hyperspectral images. According to the method, the spectral information of the hyperspectral image is fully utilized, the ground objects can be classified under the condition of a certain amount of samples, the number of parameters in the feature extraction process is effectively reduced, and the Hughes phenomenon is effectively resisted. The model provided by the invention can fully utilize the spectral characteristics of the hyperspectral image and effectively relieve the problems of intra-high-class difference and inter-high-class similarity of the pixels in the hyperspectral image in other methods.

Description

Hyperspectral image classification method based on multilevel feature extraction network
Technical Field
The invention relates to a hyperspectral image classification method.
Background
The hyperspectral imaging is one of the most important detection means in the field of remote sensing, and can obtain the spectral information of various ground objects while obtaining the ground surface distribution information, thereby realizing the combination of images and spectral information. The most prominent characteristics of the hyperspectral image are fine spectral resolution and rich spectral information, so that diagnostic characteristics different from the traditional multispectral or the characteristics found by human eyes can be effectively detected, and the cognitive ability of human beings to the world is remarkably improved. Therefore, the hyperspectral image is widely applied to the field of remote sensing image classification. The hyperspectral image classification technology is utilized, the ground environment transformation can be monitored, and therefore powerful data support is provided for ecological protection; the growth condition of the surface crops can be obtained, and real-time image data are provided for agricultural development; more, hyperspectral image classification is applied to the fields of mineral resource surveying, surface building identification and the like.
The traditional hyperspectral image classification methods include a support vector machine, a Markov random field, a decision tree and the like, most of the methods depend on feature information of artificial marks, and deep information in original data is difficult to obtain. With the development of deep learning technology, the convolutional neural network has great potential and advantages in the field of image processing. The convolutional neural network can autonomously learn and fit various features of the image to form an accurate representation of the image. At present, a convolutional neural network is widely applied to the field of remote sensing image classification, but due to the inherent spectral heterogeneity, Hughes and other problems of hyperspectral images, the conventional hyperspectral image classification method based on the convolutional neural network is difficult to fully utilize the spectral information of the hyperspectral images, and the Hughes phenomenon is serious under the condition that a small number of samples are difficult to deal with.
Disclosure of Invention
The purpose of the invention is: the method overcomes the limitation that a feature extraction model needs to be manually marked in the traditional image classification, realizes full utilization of spectral information of the hyperspectral image under a certain amount of samples through a convolutional neural network, and improves the representation of captured features, thereby effectively improving the ground object classification effect of the hyperspectral image.
In order to achieve the above object, the present invention provides a hyperspectral image classification method based on a multilevel feature extraction network, which is characterized in that a multilevel joint feature extraction network is constructed, the multilevel joint feature extraction network acquires the most distinctive features on each wave band of a hyperspectral pixel with a lightweight framework, and converts the features into global attention details for further joint feature extraction to obtain deep semantic features, thereby improving the classification effect of hyperspectral images, and specifically comprises the following steps:
step 1, constructing a multi-level combined feature extraction network, wherein the multi-level combined feature extraction network comprises two channel-by-channel multi-scale feature extraction modules, two combined attention detail fusion modules, two point-by-point convolution units and a feature fusion and decision module;
step 2, generating a data set:
based on the existing mark data in the hyperspectral image, filling the hyperspectral image in an edge mirror image filling mode to generate a data block with a mark pixel as a center, and dividing the obtained data block into a training sample set, a verification sample set and a test sample set in proportion;
step 3, sending the training sample set into the multi-level combined feature extraction network constructed in the step 1, normalizing the input training sample through a regularization layer to obtain a regularized training sample P',
Figure BDA0003347038330000021
s multiplied by S is the size of a data block in space, N is the batch number of training samples, and C represents that a hyperspectral image has C spectral bands;
step 4, sending the training sample P' into a channel-by-channel multi-scale feature extraction module to obtain feature information on each channel in the input data, and the method comprises the following steps:
step 401, sending data into a two-dimensional depth convolution layer with a size of 3 × 3 and an expansion rate of 1 and a two-dimensional depth convolution layer with a size of 3 × 3 and an expansion rate of 2 simultaneously to obtain multi-scale context information more focused on a central pixel point, and obtaining two feature maps with the same size;
step 402, respectively regularizing and adding the two obtained feature maps with the same size, then activating by using a linear correction unit function, sending the activated feature information into a standard convolution layer with the size of 3 multiplied by 3 and regularizing to obtain a feature map DC3；
Step 403, feature map DC3The distribution of key information on each channel in the input data is strengthened through residual mapping, and further information interaction between the channels is realized through a point-by-point convolution unit to obtain a characteristic diagram FpwUsing activation function to map feature FpwFeature map F obtained after activation3，
Figure BDA0003347038330000022
Feature map F3Will be sent to the feature fusion and decision module;
step 5, obtaining the characteristic diagram F obtained in the step 4pwAnd sending to a joint attention detail fusion module:
the combined attention detail fusion module firstly carries out the feature map FpwSending the space spectrum feature into a sigmoid function, converting the extracted space spectrum feature into original input combined attention detail distribution, multiplying the space spectrum feature by the training sample P 'in the step 3, and injecting the multiplied space spectrum feature into the training sample P' to obtain feature information of noise suppression information so as to obtain the training sampleP′1(ii) a Training sample P 'was then extracted with a 3X 7 sized three-dimensional convolutional layer'1Combined characteristic F of4，
Figure BDA0003347038330000031
S ' x S ' is the size of the combined feature in space, C ' represents the corresponding channel size;
step 6, combining characteristics F obtained in the step 54Extracting the deep semantic features by the same steps of the step 4 and the step 5 to obtain the combined features F output by the repeated step 55And feature map F output by using repeated step 4036，
Figure BDA0003347038330000032
Step 7, converting the characteristic diagram F3、F5And F6Meanwhile, global pooling operation on a spatial scale is carried out and is sent into a feature fusion and decision module; in the feature fusion and decision module, the feature graph F is passed3、F5And F6Splicing the obtained three eigenvectors to obtain a multi-level eigenvector F7，
Figure BDA0003347038330000033
Feature vector F7The feature vectors are sent into two full-connection layers for information fusion, and finally, the fused feature vectors are sent into a softmax classifier to obtain a final classification result;
step 8, training the network
Combining the classification result obtained in the step 7 with the real category of the training sample, and constructing a loss function of the multi-level combined feature extraction network through cross entropy; meanwhile, carrying out L2 regularization constraint on the parameters of the first full-connection layer in the step 7, and adding a Dropout layer to prevent an overfitting phenomenon in the training process; finally, repeating the steps from the step 3 to the step 7 through a back propagation algorithm to carry out multiple iterations so as to obtain the best classification effect;
in the training process, model parameters are adjusted through a verification set; after an ideal classification effect is achieved, the model is evaluated through a test set;
and 9, generating a data block by all pixels in the original hyperspectral image obtained in real time in the same manner in the step 2, sending the data block into the multi-level combined feature extraction network trained in the step 8 for classification and prediction, and finally obtaining the classified hyperspectral image.
According to the method, the spectral information of the hyperspectral image is fully utilized, the ground objects can be classified under the condition of a certain amount of samples, the number of parameters in the feature extraction process is effectively reduced, and the Hughes phenomenon is effectively resisted. Through the channel-by-channel multi-scale feature extraction module, the most characteristic feature information on each spectral channel of the hyperspectral image can be effectively obtained, and a basis is provided for further finding out key information in the hyperspectral image; through the combined attention detail fusion module, the invention converts the existing global features into the combined attention details in a concomitant mode, and inhibits redundant information in input data under the condition of not increasing training parameters so as to obtain a deeper spatial spectrum combined feature. The model provided by the invention can fully utilize the spectral characteristics of the hyperspectral image and effectively relieve the problems of intra-high-class difference and inter-high-class similarity of the pixels in the hyperspectral image in other methods.
Drawings
FIG. 1 is a schematic illustration of a process according to the present invention;
FIG. 2 is a flow chart of a method of the present invention;
FIG. 3 is a diagram of an embodiment of the method of the present invention, wherein a) a Kennedy center hyperspectral pseudo-color image, b) a Kennedy center dataset sample class label, c) a Salinaus hyperspectral pseudo-color image, d) a Salinaus dataset sample class label, e) a university of Pavea hyperspectral pseudo-color image, f) a university of Pavea dataset sample class label;
FIG. 4 is a result graph obtained by the method of the present invention on the example graph, wherein a-1) to a-9) are the classification results of Kennedy hyperspectral images, b-1) to b-9) are the classification results of Salinaus hyperspectral images, and c-1) to c-9) are the classification results of the university of Pavea hyperspectral images.
Detailed Description
The invention is further illustrated by the following specific examples. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and such equivalents may fall within the scope of the present invention as defined in the appended claims.
Taking a salenas hyperspectral image acquired by an aeronautical and astronautic office Airborne Visible/Infrared spectroscopy imager (AVIRIS) of the united states at the valley of salenas, california as an example, the image contains 204 spectral bands after removing noise bands, covers spectral information in the range of 0.4-2.5 μm, has a space size of 512 × 217, and has a spectral resolution and a spatial resolution of 10nm and 3.7 m. In the whole image, a total of 54129 image elements are marked into 16 classes, wherein: 5% of the pels were used as training samples, 2.5% as validation samples, and the remainder as test samples.
The hyperspectral image classification method based on the multilevel feature extraction network provided by the invention specifically comprises the following steps:
1) constructing a multi-level combined feature extraction network, wherein the specific structure is shown in figure 1;
2) let the original hyperspectral image be
Figure BDA0003347038330000051
M marked pixels in the H multiplied by W pixels are divided into T categories according to the characteristics of C spectral bands of the M marked pixels. Based on M marked pixels, generating a three-dimensional data block with the marked pixels as the center in a mode of edge mirror image filling
Figure BDA0003347038330000052
Where S × S is the size of the data block in space. Dividing the obtained three-dimensional data block P into a training sample set P according to the proportion stated in the foregoingtrainVerifying the sample set PvalAnd test sample set PtestThat is, 5% of the three-dimensional data blocks P are used as training samples, 2.5% of the three-dimensional data blocks P are used as verification samples, and the remaining three-dimensional data blocks P are used as test samples.
3) Will train the sample set PtrainSending the multi-level combined feature extraction network constructed in the step 1). In the multi-level combined feature extraction network, training samples are normalized through a regularization layer to obtain regularized training samples
Figure BDA0003347038330000053
Where N represents the number of batches of training samples, in this embodiment, N is 32. The training sample P' corresponds to a unique hot code label vector set of
Figure BDA0003347038330000054
The correlation calculation is shown in the following formula (1):
P′＝ο(Ptrain) (1)
in expression (1), o (-) is a regularization operation.
4) Sending the training sample P' into a channel-by-channel multi-scale feature extraction module to obtain feature information on each channel in the training sample data, and specifically comprising the following steps:
401) simultaneously sending training sample data into a two-dimensional depth convolution layer with the size of 3 multiplied by 3 and the expansion rate of 1 and a two-dimensional depth convolution layer with the size of 3 multiplied by 3 and the expansion rate of 2 so as to obtain multi-scale context information which is more concentrated on the central pixel point;
402) respectively carrying out regularization treatment on two feature maps with the same size obtained by two-dimensional depth convolution layers and then adding the feature maps;
403) activation is performed by using a Linear correction Unit (ReLU) function, which is specifically calculated as shown in the following formula (2):
DC1＝W1*P′+b1
DC2＝W2*P′+b2 (2)
F1＝δ(ο(DC2)+ο(DC2))
in the formula (2), W1、W2Convolution kernels of the normal depth convolution and the depth convolution with a dilation rate of 2, b, respectively1、b2Is a corresponding filter bias parameter, DC1、DC2Is a two feature map of equal size, δ (-) is the ReLU activation function, F1Is the final output feature map;
404) activated feature map F1Sending the data into a standard convolution layer with the size of 3 multiplied by 3 and carrying out regularization treatment; then, the characteristic diagram DC subjected to the characteristic processing3The distribution of key information on each channel in the input data is strengthened through residual mapping, and further information interaction between channels is realized by using a point-by-point convolution unit, and the specific calculation is as shown in the following formula (3):
Figure BDA0003347038330000064
in the formula (3), W3、b3Convolution kernels of a standard 3 x 3 sized convolution layer and corresponding filter bias parameters, F2Corresponding intermediate calculation result, FpwFeature maps output for point-by-point convolution, Wpw、bpwThe convolution kernel and corresponding filter bias parameters for the point-by-point convolution,
Figure BDA0003347038330000061
is a feature map to be fed into a feature fusion and decision module;
5) the characteristic diagram F obtained in the step 4) is usedpwSending into a joint attention detail fusion module, wherein in the joint attention detail fusion module:
the obtained characteristic diagram FpwSending a sigmoid function, converting the extracted spatial spectrum characteristics into original input combined attention detail distribution, multiplying the spatial spectrum characteristics by the training sample P 'in the step 3), and injecting the multiplied spatial spectrum characteristics into the training sample P' to obtain characteristic information for inhibiting noise information; further, the obtained features are extracted using a three-dimensional convolution layer of 3X 7 sizeThe joint feature of the feature information is specifically calculated as shown in the following formula (4):
Figure BDA0003347038330000063
in the formula (4), sigmoid function sigma (-) converts feature map FpwScaling to [0,1 ]]Within range, convert to global attention detail distribution A and multiply by element
Figure BDA0003347038330000062
Injecting into P 'to obtain P'1Further obtaining a combined feature F by a three-dimensional convolution operation4，W4、b4A convolution kernel that is a standard 3 x 3 sized three-dimensional convolution and corresponding filter bias parameters;
6) combining the characteristics F obtained in step 5)4Extracting deep semantic features to obtain combined features F by the same steps of step 4) and step 5)5Output characteristic diagram F of another point-by-point convolution unit6；
7) Will feature map
Figure BDA0003347038330000071
And simultaneously performing global pooling operation on a spatial scale, and sending the global pooling operation into a feature fusion and decision module, wherein S '× S' and S '× S' are the sizes of the data blocks in the corresponding space, and C 'and C' represent the sizes of the corresponding features in the channel dimension.
In the feature fusion and decision module, the obtained three feature vectors are spliced to obtain a multi-level feature vector F7The following formula (5) is calculated:
Figure BDA0003347038330000072
in the formula (5), the reaction mixture is,
Figure BDA0003347038330000073
and is respectively the global convolution pooling operation and directionAnd (4) measuring splicing operation.
Feature vector F7Will be sent to two full connection layers for information fusion to obtain fused feature vector F8The following formula (6) is calculated:
Figure BDA0003347038330000074
in the formula (6), the reaction mixture is,
Figure BDA0003347038330000075
is a binary distance; λ is a penalty factor, which is set to 0.02 in this embodiment, for the parameter of the first fully-connected layer
Figure BDA0003347038330000077
Performing an L2 regularization constraint; φ (-) is a Dropout layer operation with the mask rate set to 0.4; is a matrix multiplication;
Figure BDA0003347038330000078
is the output of the second fully connected layer;
Figure BDA0003347038330000079
a filter bias parameter representing a first fully-connected layer;
Figure BDA00033470383300000710
representing the filter bias parameters of the second fully connected layer.
Fused feature vector F7Will be sent to the softmax classifier to obtain the final classification result
Figure BDA0003347038330000076
The calculation is specifically shown in the following formula (7):
Figure BDA0003347038330000081
Figure BDA0003347038330000082
Figure BDA0003347038330000083
in the formula (7), the reaction mixture is,
Figure BDA0003347038330000084
is a set of prediction vectors F8The ith prediction vector;
Figure BDA0003347038330000085
is a reaction of with F8 iThe output vector corresponding to (softmax-;
Figure BDA0003347038330000086
is the softmax (-) output vector for the batch of training samples; prediction is the classification prediction result of the final batch of training samples obtained by the argmax (·) function.
8) And training the network.
Combining the classification result obtained in the step 7) with the real category of the training sample, constructing a loss function of the network through cross entropy, and calculating as shown in the following formula (8):
Figure BDA0003347038330000087
in the formula (8), the reaction mixture is,
Figure BDA00033470383300000810
is the real label Y corresponding to the ith sample in the batch sampleiThe value of the t-th value of (c),
Figure BDA0003347038330000088
is the corresponding prediction tag;
and finally, repeating the steps from the step 3) to the step 7) for batch multiple iterations through a back propagation algorithm and an Adam random gradient descent method to obtain the best classification effect. During the training process, the validation set will be used to adjust the model parameters. After the desired classification effect is achieved, the test set is used to evaluate the model.
9) Generating data blocks of all pixels in the original hyperspectral image in the same way in the step 2)
Figure BDA0003347038330000089
And sending the hyperspectral images into the network trained in the step 8) for classification and prediction, and finally obtaining the classified hyperspectral images
Fig. 3 shows an embodiment of the present invention, in which a) is a hyperspectral pseudo-color image of kennedy center, b) is a schematic diagram of class labels for marking pixels, the data set is obtained by AVIRIS equipment over kennedy space center in the united states, and after removing the low signal-to-noise ratio band, the image contains 176 bands, and the spectral information covering the range of 0.4-2.5 μm, and the spatial size is that the spectral resolution and the spatial resolution are 10nm and 18 m. In the whole image, 5211 pixels are marked and classified into 13 types, wherein 15% of the pixels are used as training samples, 5% of the pixels are used as verification samples, and the rest pixels are used as test samples; fig. c) is a hyperspectral false color image of salinus, d) is a schematic diagram of a category label of a marked pixel, and detailed information is described in the foregoing; fig. e) is a hyperspectral pseudo-color image of houston university, f) is a class label schematic diagram of a labeled pixel thereof, the data set is obtained by a Reflective optical System Imaging Spectrometer (rosss) over the university of italian paviia, the image comprises 103 bands, spectral information covering the range of 0.43-0.86 μm, and the spatial dimensions are such that the spectral resolution and the spatial resolution are divided into 3.74nm and 1.3 m. In the whole image, 42766 pixels are marked and classified into 9 types, wherein 5% of the pixels are used as training samples, 2.5% of the pixels are used as verification samples, and the rest pixels are used as test samples.
The method designed by the invention is compared with 8 hyperspectral image fusion methods, namely SVM (support vector machine-based classification method), EMP (extended morphological contour-based classification method), hybrid SN (hybrid convolutional network-based classification method), RSSAN (residual spectrum-air attention network-based classification method), JSAN (joint air-spectrum attention network-based classification method), SSSERN (air-spectrum feature compression and excitation residual error network-based classification method), TSCNN (double-current convolutional neural network-based classification method of depth feature fusion network), FG-SSCA (serial air-spectrum attention mechanism and feature packet network-based classification method), wherein the classification results of Kennedy data are shown in table 1, the classification results of Sanaissay data are shown in table 2, the classification results of the university of Pavea are shown in table 3, wherein the data evaluation standard is Overall Accuracy (Overallaccuracy, OA), Average Accuracy (AA), and Kappa coefficient (Kappa coefficient).
Table 1 kennedy dataset classification evaluation results (%)
Figure BDA0003347038330000091
Figure BDA0003347038330000101
TABLE 2 Classification evaluation results (%)
Figure BDA0003347038330000102
Table 3 university of parkia data set classification evaluation results (%)
Figure BDA0003347038330000103
Figure BDA0003347038330000111
In the eight methods, the relevant parameters of the first two traditional methods are obtained by manual debugging, and the relevant parameters of the other six deep learning-based classification methods are selected according to the prompts of the relevant papers; in the method provided by the invention, the spatial size of the data block is set as 0.0003 for the learning rate and 0.9 for the momentum, and meanwhile, the attenuation coefficient of each 100 iterations of the learning rate is set as 0.0005, the batch number is set as 32, and 300 iterations are performed. The result shows that the method designed by the invention has better classification effect. The prediction accuracy of the classification method on each category of each data set also shows the effectiveness of the method, the spectral features of the hyperspectral images can be fully utilized by the method provided by the invention, and the distinguishable multi-level combined features are obtained for classification, and to a certain extent, the method can relieve the challenges brought by spectral heterogeneity under a small amount of data.
Fig. 4 is a classification result image of the present invention and other methods, and it can be seen from the figure that compared with other fusion methods, the method provided by the present invention can better overcome the phenomenon of spectral heterogeneity, and obtain a good classification image.
Although the present invention has been described with reference to the preferred embodiments, it is not intended to be limited thereto, and those skilled in the art can make possible variations and modifications of the present invention using the above-described methods and techniques without departing from the spirit and scope of the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are within the protection scope of the technical solution of the present invention, unless the content of the technical solution of the present invention is departed from.

Claims (1)

1. The method for classifying the hyperspectral images based on the multilevel feature extraction network is characterized in that the multilevel joint feature extraction network is constructed, the multilevel joint feature extraction network acquires the most distinctive features on each wave band of a hyperspectral pixel by a lightweight framework, converts the distinctive features into global attention details for further joint feature extraction, obtains deep semantic features, and further improves the classification effect of the hyperspectral images, and specifically comprises the following steps:
step 1, constructing a multi-level combined feature extraction network, wherein the multi-level combined feature extraction network comprises two channel-by-channel multi-scale feature extraction modules, two combined attention detail fusion modules, two point-by-point convolution units and a feature fusion and decision module;
step 2, generating a data set:
based on the existing mark data in the hyperspectral image, filling the hyperspectral image in an edge mirror image filling mode to generate a data block with a mark pixel as a center, and dividing the obtained data block into a training sample set, a verification sample set and a test sample set in proportion;
step 3, sending the training sample set into the multi-level combined feature extraction network constructed in the step 1, normalizing the input training sample through a regularization layer to obtain a regularized training sample P',
Figure FDA0003347038320000011
s multiplied by S is the size of a data block in space, N is the batch number of training samples, and C represents that a hyperspectral image has C spectral bands;
step 4, sending the training sample P' into a channel-by-channel multi-scale feature extraction module to obtain feature information on each channel in the input data, and the method comprises the following steps:
step 401, sending data into a two-dimensional depth convolution layer with a size of 3 × 3 and an expansion rate of 1 and a two-dimensional depth convolution layer with a size of 3 × 3 and an expansion rate of 2 simultaneously to obtain multi-scale context information more focused on a central pixel point, and obtaining two feature maps with the same size;
step 402, respectively regularizing and adding the two obtained feature maps with the same size, then activating by using a linear correction unit function, sending the activated feature information into a standard convolution layer with the size of 3 multiplied by 3 and regularizing to obtain a feature map DC3；
Step 403, feature map DC3The distribution of key information on each channel in the input data is strengthened through residual mapping, and further information interaction between the channels is realized through a point-by-point convolution unit to obtain a characteristic diagram FpwBy using laserLive function graph FpwFeature map F obtained after activation3，
Figure FDA0003347038320000012
Feature map F3Will be sent to the feature fusion and decision module;
step 5, obtaining the characteristic diagram F obtained in the step 4pwAnd sending to a joint attention detail fusion module:
the combined attention detail fusion module firstly carries out the feature map FpwSending the space spectrum feature into a sigmoid function, converting the extracted space spectrum feature into original input combined attention detail distribution, multiplying the space spectrum feature by the training sample P 'in the step 3, and injecting the multiplied space spectrum feature into the training sample P' to obtain feature information of noise suppression information so as to obtain the training sample P1'; the training sample P is then extracted using a three-dimensional convolution layer of size 3X 71' combination feature F4，
Figure FDA0003347038320000021
S ' x S ' is the size of the data block in space, C ' represents the corresponding channel size;
step 6, combining characteristics F obtained in the step 54Extracting the deep semantic features by the same steps of the step 4 and the step 5 to obtain the combined features F output by the repeated step 55And feature map F output by using repeated step 4036，
Figure FDA0003347038320000022
Step 7, converting the characteristic diagram F3、F5And F6Meanwhile, global pooling operation on a spatial scale is carried out and is sent into a feature fusion and decision module; in the feature fusion and decision module, the feature graph F is passed3、F5And F6Splicing the obtained three eigenvectors to obtain a multi-level eigenvector F7，
Figure FDA0003347038320000023
Feature vector F7The feature vectors are sent into two full-connection layers for information fusion, and finally, the fused feature vectors are sent into a softmax classifier to obtain a final classification result;
step 8, training the network
Combining the classification result obtained in the step 7 with the real category of the training sample, and constructing a loss function of the multi-level combined feature extraction network through cross entropy; meanwhile, carrying out L2 regularization constraint on the parameters of the first full-connection layer in the step 7, and adding a Dropout layer to prevent an overfitting phenomenon in the training process; finally, repeating the steps from the step 3 to the step 7 through a back propagation algorithm to carry out multiple iterations so as to obtain the best classification effect;
in the training process, model parameters are adjusted through a verification set; after an ideal classification effect is achieved, the model is evaluated through a test set;
and 9, generating a data block by all pixels in the original hyperspectral image obtained in real time in the same manner in the step 2, sending the data block into the multi-level combined feature extraction network trained in the step 8 for classification and prediction, and finally obtaining the classified hyperspectral image.
CN202111326280.1A 2021-11-10 2021-11-10 Hyperspectral image classification method based on multilevel feature extraction network Pending CN114048810A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111326280.1A CN114048810A (en) 2021-11-10 2021-11-10 Hyperspectral image classification method based on multilevel feature extraction network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111326280.1A CN114048810A (en) 2021-11-10 2021-11-10 Hyperspectral image classification method based on multilevel feature extraction network

Publications (1)

Publication Number Publication Date
CN114048810A true CN114048810A (en) 2022-02-15

Family

ID=80207998

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111326280.1A Pending CN114048810A (en) 2021-11-10 2021-11-10 Hyperspectral image classification method based on multilevel feature extraction network

Country Status (1)

Country Link
CN (1) CN114048810A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116563571A (en) * 2023-05-16 2023-08-08 北京师范大学 Boltzmann entropy similarity-based hyperspectral image band selection method and system
CN116595208A (en) * 2023-07-17 2023-08-15 云南大学 Classification method and device for hyperspectral images and electronic equipment
CN117115668A (en) * 2023-10-23 2023-11-24 安徽农业大学 Crop canopy phenotype information extraction method, electronic equipment and storage medium

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116563571A (en) * 2023-05-16 2023-08-08 北京师范大学 Boltzmann entropy similarity-based hyperspectral image band selection method and system
CN116563571B (en) * 2023-05-16 2023-11-21 北京师范大学 Boltzmann entropy similarity-based hyperspectral image band selection method and system
CN116595208A (en) * 2023-07-17 2023-08-15 云南大学 Classification method and device for hyperspectral images and electronic equipment
CN116595208B (en) * 2023-07-17 2023-10-13 云南大学 Classification method and device for hyperspectral images and electronic equipment
CN117115668A (en) * 2023-10-23 2023-11-24 安徽农业大学 Crop canopy phenotype information extraction method, electronic equipment and storage medium
CN117115668B (en) * 2023-10-23 2024-01-26 安徽农业大学 Crop canopy phenotype information extraction method, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
Kang et al. Classification of hyperspectral images by Gabor filtering based deep network
CN110443143B (en) Multi-branch convolutional neural network fused remote sensing image scene classification method
Gao et al. Spectral superresolution of multispectral imagery with joint sparse and low-rank learning
Qu et al. uDAS: An untied denoising autoencoder with sparsity for spectral unmixing
Bergado et al. Recurrent multiresolution convolutional networks for VHR image classification
CN109145992B (en) Hyperspectral image classification method for cooperatively generating countermeasure network and spatial spectrum combination
Nandhini Abirami et al. Deep CNN and deep GAN in computational visual perception-driven image analysis
Peng et al. Self-paced nonnegative matrix factorization for hyperspectral unmixing
CN109766858A (en) Three-dimensional convolution neural network hyperspectral image classification method combined with bilateral filtering
CN114048810A (en) Hyperspectral image classification method based on multilevel feature extraction network
CN103971123B (en) Hyperspectral image classification method based on linear regression Fisher discrimination dictionary learning (LRFDDL)
Li et al. Structure-aware collaborative representation for hyperspectral image classification
Basu et al. A semiautomated probabilistic framework for tree-cover delineation from 1-m NAIP imagery using a high-performance computing architecture
CN107145836B (en) Hyperspectral image classification method based on stacked boundary identification self-encoder
CN107563442B (en) Hyperspectral image classification method based on sparse low-rank regular graph tensor embedding
CN113723255B (en) Hyperspectral image classification method and storage medium
CN111639587B (en) Hyperspectral image classification method based on multi-scale spectrum space convolution neural network
Sun et al. Extracting urban impervious surface from worldView-2 and airborne LiDAR data using 3D convolutional neural networks
CN112347888A (en) Remote sensing image scene classification method based on bidirectional feature iterative fusion
CN111783884B (en) Unsupervised hyperspectral image classification method based on deep learning
CN112580480B (en) Hyperspectral remote sensing image classification method and device
CN102629374A (en) Image super resolution (SR) reconstruction method based on subspace projection and neighborhood embedding
CN113139512B (en) Depth network hyperspectral image classification method based on residual error and attention
CN115187861A (en) Hyperspectral image change detection method and system based on depth twin network
Wu et al. A Siamese template matching method for SAR and optical image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination