CN114882368A - Non-equilibrium hyperspectral image classification method - Google Patents
Non-equilibrium hyperspectral image classification method Download PDFInfo
- Publication number
- CN114882368A CN114882368A CN202210626027.6A CN202210626027A CN114882368A CN 114882368 A CN114882368 A CN 114882368A CN 202210626027 A CN202210626027 A CN 202210626027A CN 114882368 A CN114882368 A CN 114882368A
- Authority
- CN
- China
- Prior art keywords
- layer
- hyperspectral image
- training
- dimensional
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a depth-generated spectrum-space classifier-based unbalanced hyperspectral image classification method, which mainly solves the problem that the classification precision of small categories in unbalanced hyperspectral classification tasks is poor in the prior art. The implementation scheme is as follows: acquiring a hyperspectral image, selecting wave bands, and dividing the hyperspectral image into a training sample and a test sample; constructing a classification network of the unbalanced hyperspectral image, wherein the classification network comprises a two-stage three-dimensional encoder, a three-dimensional decoder, a small-class upsampling module and a classifier; training the classification network by using a training sample, setting a total loss function, initializing a network training parameter, and updating the classification network by adopting a gradient descent method until the maximum iteration number is reached; and inputting the test sample into the trained classification network to obtain a classification result. The method improves the classification precision of small categories in the hyperspectral image, enhances the robustness, and can be used for mineral exploration, ecological monitoring, intelligent agriculture and medical diagnosis.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a non-equilibrium hyperspectral image classification method which can be used for mineral exploration, ecological monitoring, intelligent agriculture and medical diagnosis.
Background
The hyperspectral image has high spectral resolution, can cover visible light, near infrared and short-wave infrared wavelength ranges, and generally has tens of to hundreds of wave bands. The unique and fine spectral information of different substances makes it possible to distinguish land cover categories with only slight differences, and is widely applied in mineral exploration, ecological monitoring, intelligent agriculture, medical diagnosis and the like. The hyperspectral image classification is one of important means for hyperspectral image interpretation, and aims to endow each pixel point in an image with a specific label, so that pixel-level semantic analysis of a scene is realized, and powerful support can be provided for higher-level applications such as ground object monitoring and change detection.
In the actually shot hyperspectral images, the difference of the area proportion occupied by different types of ground objects in the images is large, so that sample data used for training the classifier presents an unbalanced distribution rule, namely, some type samples are large in number and belong to large types, and some type samples are small in number and belong to small types. Under the above circumstances, the training process of the classifier will depend heavily on the labeling information of the large-class samples, and pay less attention to the small-class samples, so that the classification result is biased to the large class, and the classification accuracy of the small class is poor. However, a few classes in a scene are generally more valuable, requiring a high quality recognition rate. For example, in the classification of vegetation-covered scenes, precious and rare species of more interest typically occupy only a small fraction of the pixels in an image; in real-time classification application, a forest fire only occupies a small area in an image scene at the initial stage, and a classifier is required to accurately classify the small category as soon as possible, so that timely disaster prevention and reduction are realized. Therefore, the design of the unbalanced hyperspectral image classification method has very important significance and value.
The traditional method for solving the classification of the unbalanced hyperspectral images directly resamples original sample data, namely respectively upsampling and downsampling small-class samples and large-class samples, so that the number of the samples of all classes is balanced. The method aims at the relatively low-dimensional sample characteristics, namely the one-dimensional spectral curve corresponding to the sample, and is difficult to process high-dimensional data such as three-dimensional spectrum-space neighborhood samples. The deep learning model can be driven by original data, and high-dimensional data can be effectively mapped to a low-dimensional feature embedding space with stronger distinguishing capability. With the introduction of the method, researchers introduce a resampling strategy into a deep learning model, capture data distribution in an embedding space by training a deep network, and synthesize a new sample by taking the embedding space as a bridge. The conditional variation self-encoder captures the dependency relationship of data by using the implicit characteristics extracted by the network, can generate samples of a specific class from the learned latent variable distribution by taking the label information as a condition, and does not need a complex antagonistic learning process. However, if the conditional variational self-encoder is directly used to synthesize samples, the generation and classification of data are processes isolated from each other, resulting in poor classification effect. In addition, the conditional variational self-encoder is generally used for generating a simple one-dimensional sequence signal or a two-dimensional natural image, but not high-dimensional hyperspectral space spectrum neighborhood sample data, so that rich spectrum-space information cannot be fully utilized.
Swalpa Kumar Roy et al, in its published article "Generation adaptive optimization for Spectral-spatial Hyperspectral Image Classification" ("IEEE Transactions on Geoscience & Remote Sensing", 2022, 60:1-15), proposed a small class upsampling method (HyperGAMO) based on Generative countermeasure network, which was used to overcome the problems in unbalanced Hyperspectral Classification. According to the method, a three-dimensional convex hull generator for generating the countermeasure network based on conditions is designed, and new sample characteristics are generated in a convex hull formed by the depth characteristics of the small-class samples, so that the processes of sample generation and classification are realized in an end-to-end mode, and the complexity of a deep generation model for a classification task is reduced. Nevertheless, the method still has the defects that the small class convex hull is far from the real data distribution, so that the useful information amount of the synthesized sample is small, the problem of insufficient small class samples cannot be solved, and the classification performance needs to be further improved.
Disclosure of Invention
The invention aims to provide a non-equilibrium hyperspectral image classification method based on a depth-generated spectrum-space classifier, aiming at overcoming the defects of the prior art, so that the spectrum-space information is fully utilized, the quality and the quantity of subclass samples are enhanced, and the image classification effect is improved.
The technical idea for realizing the aim is as follows: the method combines data generation and classification into a unified process, fully excavates hyperspectral image information through an encoder, and performs upsampling on small classes in a hidden space, so that a generated sample is closer to a real sample, a more accurate data enhancement effect is obtained, and the classification precision is further improved.
According to the above-mentioned idea the invention includes the following steps:
1) acquiring hyperspectral image data, selecting wave bands, segmenting the hyperspectral image after the wave band selection into spectrum-space characteristic neighborhood blocks according to a fixed size, and then dividing the spectrum-space characteristic neighborhood blocks according to the ratio of 1: 99 into training samples and test samples;
2) constructing an unbalanced hyperspectral image classification network model:
2a) establishing a three-dimensional encoder which is formed by three-dimensional convolution modules and two-dimensional convolution modules which are cascaded and are cascaded with two parallel fully-connected networks and then cascaded with a re-parameter module;
2b) setting a small-category up-sampling module in an implicit space;
2c) establishing a three-dimensional decoder consisting of 1 full-connection layer and a plurality of deconvolution layers;
2d) establishing an image classifier module consisting of 1 full connection layer;
2e) cascading a three-dimensional encoder and a small-category up-sampling module, connecting a three-dimensional decoder and a classifier in parallel, and simultaneously inputting the output of the small-category up-sampling module into the three-dimensional decoder and the classifier in parallel to form an unbalanced hyperspectral image classification network model;
3) training a non-equilibrium hyperspectral image classification network model:
3a) constructing the overall loss function L of the network model Total ＝λ·L mmd +L PdRec +L cls Wherein L is mmd Is a regular term loss function based on maximizing the mean difference, L PdRec As a reconstruction loss function based on the neighborhood distance, L cls Is a classification loss function based on cross entropy, and lambda is a parameter;
3b) the parameters for initializing the unbalanced hyperspectral image classification network model comprise a weight parameter w, a bias parameter b and a parameter theta in a three-dimensional encoder, wherein the parameter theta is ═ theta 1 ,θ 2 Weight parameter w, bias parameter b and parameter in three-dimensional decoder
Mean value parameter in subclass sample characteristic up-sampling module
Sum variance parameter
And a classifier parameter η;
3c) inputting the training samples into the unbalanced hyperspectral image classification network model, circularly updating parameters in the unbalanced hyperspectral image classification network by adopting a gradient descent method to reduce the gradient value of the total loss function until the maximum iteration number T is reached, and obtaining the trained unbalanced hyperspectral image classification network model;
4) and inputting the test sample into a trained unbalanced hyperspectral image classification network model, and outputting a hyperspectral image classification result graph.
The invention has the beneficial effects that:
firstly, due to the fact that the unbalanced hyperspectral image classification network comprising the three-dimensional encoder, the small-class upsampling module, the three-dimensional decoder and the classifier in two stages is built, input data can directly obtain a result of an output end through the trained unbalanced hyperspectral image classification network, manual preprocessing and subsequent processing are avoided, and the hyperspectral image classification speed is improved.
Secondly, compared with the traditional method of calculating the reconstruction loss function by using the Euclidean distance to measure the characteristics of the one-dimensional signal or the two-dimensional image, the reconstruction loss function based on the neighborhood distance designed by the invention effectively improves the consistency between the three-dimensional sample generated after decoding and the input three-dimensional sample, and improves the overall classification precision of the classifier.
Thirdly, the invention captures the data distribution of the hidden features in the embedding space to realize the resampling of the small-class samples, thereby integrating the deep spectrum-space feature generation and the classifier into a unified framework, and enabling the classifier to have higher precision and stronger robustness particularly on the small-class data under the condition of processing unbalanced data.
Compared with the prior art, the method can better utilize the spectrum-space integrated characteristic information of the hyperspectral image, can keep the overall classification precision at a higher level under the condition of unbalanced samples, can effectively improve the classification performance of small sample classes, and can provide better services for higher-level applications such as ground object monitoring and change detection.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a block diagram of a non-equilibrium hyperspectral image classification model constructed in the present invention;
FIG. 3 is a hyperspectral image taken from a public website;
FIG. 4 is a diagram of simulation results from classifying FIG. 3 using the prior HyperGAMO method;
FIG. 5 is a graph of simulation results for classifying FIG. 3 using the present invention.
Detailed Description
The embodiments and effects of the present invention will be further explained with reference to the accompanying drawings:
the example takes the University of Pavia hyperspectral classification standard dataset as an example, and classifies the dataset. The pseudo-color composite of the University of Pavia data obtained by the reflectance optical system imaging spectrometer is shown in fig. 3(a), and the corresponding truth label is shown in fig. 3(b), which contains 9 types of ground objects, for a total of 42776 labeled samples.
Referring to fig. 1, the specific implementation steps of this example are as follows:
step 1: training and testing samples were constructed.
1.1) processing the hyperspectral image by using a principal component analysis method, realizing feature dimensionality reduction and reserving the first 20 principal components;
1.2) constructing samples by adopting a principal component waveband in a mode of taking spectrum-space neighborhood characteristics with a fixed size, wherein the size of each sample is 13 multiplied by 20;
1.3) randomly selecting a proportion of 1% from the samples shown in fig. 3 as a training set, totaling 423 samples, using the rest samples as a test set, totaling 42353 samples, wherein the training set is expressed as X ═ X 1 ,...,x 423 Y ═ Y label 1 ,...,y 423 }，x i For any training sample, x is due to the co-existence of 9 classes of ground objects in the data i Corresponding label y i ∈{1,2,...,9}；
1.4) encoding the label of each training sample into a vector y in a one-hot form i 。
Step 2: and constructing a non-equilibrium hyperspectral image classification algorithm model based on the depth generation model.
Referring to fig. 2, the specific implementation of this step is as follows:
2.1) building a three-dimensional encoder f θ ：
2.1.1) three-dimensional convolution modules are arranged, wherein the first two three-dimensional convolution modules are formed by cascading a convolution layer, a standard layer and an activation layer, the last three-dimensional convolution module is formed by cascading a convolution layer, a standard layer, an activation layer and a reconstruction layer, the three standard layers and the three activation layers have the same structure and parameters, and the convolution kernel parameters of the three convolution layers are different, namely:
a first three-dimensional convolutional layer, the number of input vectors of which is 1, the size of each input vector is (13,13,20), the number of convolutional kernels is 32, the size of each convolutional kernel is (3,3,7), the step size is (1,1,1), the number of output vectors is 32, and the size of each input vector is (11,11, 14);
a second three-dimensional convolutional layer, having a number of input vectors of 32, a size of each input vector of (11,11,14), a number of convolution kernels of 64, a size of each convolution kernel of (3,3,5), a step size of (1,1,1), a number of output vectors of 64, and a size of each output vector of (9,9, 10);
and a third three-dimensional convolution module, wherein the input vector size is 64, the size of each input vector is (9,9,10), the number of convolution kernels is 128, the size of each convolution kernel is (3,3,3), the step size is (1,1,1), the number of output vectors is 128, and the size of each output vector is (7,7,8), so that the output vector sizes of 128 (7,7,8) are converted into (7,7,1024) through a reconstruction layer in the third three-dimensional convolution module, and connection with a subsequent module is facilitated.
2.1.2) a two-dimensional convolution module is arranged, the module is composed of convolution layers, a normalization layer, an activation layer and a reconstruction layer in cascade connection, the number of input vectors of the convolution layers is 1024, the size of each vector is (7,7), the number of convolution kernels is 128, the size of each convolution kernel is (3,3), the number of output vectors is 128, and the size of each vector is (5, 5).
The output of the two-dimensional convolution needs to be converted into a one-dimensional vector by a reconstruction layer therein, i.e. the output is reconstructed into a one-dimensional output vector with the size of 3200
Wherein, theta 1 Parameter, x, representing the first stage i Is the ith training sample. Two-dimensional convolution module excavates depth space features in hyperspectral sample fromAnd the spatial information contained in the hyperspectral is fully utilized.
2.1.3) setting a fully connected network, which consists of a connection hierarchy connecting two fully connected networks in parallel, wherein:
connecting layer to label y i And features of
Concatenating to cause the feature vector to obtain information of the tag;
the first fully-connected network consists of two fully-connected layers with different sizes, the conversion matrix size of the first fully-connected layer is (3209, 3200), and the conversion matrix size of the second fully-connected layer is (3200, 64);
the second fully-connected network is also composed of two fully-connected layers of different sizes, the 1 st fully-connected layer has a transition matrix size of (3209, 3200), and the 2 nd transition matrix size of (3200, 64).
2.1.4) sequentially cascading the 3 three-dimensional convolution modules, the 1 two-dimensional convolution module, the 1 full-connection module and the existing heavy parameter module to form a three-dimensional encoder;
2.2) selecting a small-class upsampling module to process the variable z in the implicit space i Up-sampling small samples of (1), the up-sampling number being
Wherein n is c Represents a sample x i The number of samples belonging to class c;
2.3) building a three-dimensional decoder
2.3.1) setting a full connection layer with size (64,4032) and output size 4032 of a conversion matrix;
2.3.2) setting a reconstruction layer for size conversion of the full connection layer output, wherein the output is 64 vectors with the size of (3,3, 7);
2.3.3) constructing three-dimensional deconvolution modules, wherein each three-dimensional deconvolution module consists of an deconvolution layer, a normalization layer and an activation layer in cascade connection, the three normalization layers and the three activation layers have the same structure, and the convolution kernel parameters of the three deconvolution layers are different, namely:
the number of input vectors of the first deconvolution layer is 64, the size of each input vector is (3,3,7), the number of convolution kernels is 32, the size of each convolution kernel is (3,3,3), the step size is (2,2,2), the number of output vector sizes is 32, and the size of each output vector is (6,6, 10);
the number of input vectors of the second deconvolution layer is 32, the size of each input vector is (6,6,10), the number of convolution kernels is 16, the size of each convolution kernel is (3,3,3), the step size is (2,2,2), the number of output vectors is 16, and the size of each output vector is (12,12, 20);
the number of input vectors of the third deconvolution layer is 16, the size of each input vector is (12,12,20), the number of convolution kernels is 1, the size is (2,2,2), the step size is (1,1,1), and the size of the output vector is (13,13, 20).
2.3.3) the 1 full-connection layer, the 1 reconstruction layer, the 1 two-dimensional convolution module and the three-dimensional deconvolution modules are sequentially cascaded to form the three-dimensional decoder.
2.3.4) three-dimensional decoder implicit features obtained in step 2.1
And the upsampled feature z obtained in step 2.2 i Generated samples reconstructed to approximate original spectral-spatial neighborhood samples
To ensure the ability of the implicit features to characterize the input sample.
2.4) construction of classifier g η 。g η Including 1 fully-connected layer, the size of the transformation matrix is (64,9) to convert the original implicit characteristics described above
And up-sampling features, converted into probability vectors belonging to various classes
For determining the class to which the sample belongs.
And 2.5) cascading the three-dimensional encoder with the small-category up-sampling module, connecting the three-dimensional decoder with the classifier in parallel, and simultaneously inputting the output of the small-category up-sampling module into the three-dimensional decoder and the classifier in parallel to form a non-equilibrium hyperspectral image classification network model.
And step 3: and constructing a loss function of the unbalanced hyperspectral image classification network model training model.
3.1) constructing a regular term loss function L based on the maximum mean difference mmd For constraining the mean μ and variance parameters σ 2 The determined consistency of the conditional probability distribution of the implicit features and the Gaussian prior distribution is as follows:
wherein p is θ (z i |x i ,y i ) A conditional probability distribution of an implicit characteristic is represented,
a gaussian prior distribution representing an implicit characteristic;
3.2) constructing a reconstruction loss function L based on the neighborhood distance PdRec For causing the reconstructed three-dimensional features to remain similar to the original sample features:
calculating a pixel p ik And
the distance between:
calculating a pixel
And x i The distance between:
using the above two distances, pixel p is calculated ik And a pixel
The distance between:
according to pixel p ik And a pixel
The distance between the two points is obtained, and a reconstruction loss function based on the neighborhood distance is obtained:
wherein d represents the calculated Euclidean distance, p ik Denotes x i Is detected in the image data of any one of the pixels,
to represent
Is detected in the image data of any one of the pixels,
is x i Correspondingly generating a sample;
3.3) selecting the existing classification loss function based on the cross entropy:
for reducing the difference between the classification result of the training sample feature and the upsampled feature and its truth label, where y ic And
x representing true and predicted respectively i Probability of belonging to class c;
3.4) use of a regularized term loss function L based on the maximum mean difference mmd And reconstructing loss function L based on neighborhood distance PdRec Cross entropy based classification loss function L cls To obtain the total loss function L of the training model total ：
L total ＝λ·L mmd +L PdRec +L cls Where λ represents a weight parameter, and oh has an example value of 0.01.
And 4, step 4: and (3) performing parameter training on the hyperspectral image classification model by using a training sample and adopting a gradient descent method.
4.1) setting an initial value alpha of a learning rate, a maximum iteration time T and a batch processing quantity B;
4.2) initializing parameters of the network model, including weight parameter w, bias parameter b and parameter θ ═ θ in the three-dimensional encoder 1 ,θ 2 Weight parameter w, bias parameter b and parameter in three-dimensional decoder
Mean value parameter in subclass sample characteristic up-sampling module
Sum variance parameter
And a classifier parameter η;
4.3) dividing the training samples into batches according to the batch processing size, and sequentially inputting the training samples into the unbalanced hyperspectral image classification network for back propagation and total loss function updating
According to a gradient of
Forward propagation is carried out in the descending direction to update the network training parameters, and the calculation is carried out on each batch of input networks until the whole training set is subjected toInputting a sample into a network for traversing to finish one training;
4.4) circulating the process of 4.3) until the maximum iteration time T is reached, and obtaining the trained unbalanced image classification model.
And 5: and classifying the test sample by using the trained unbalanced hyperspectral image classification model.
Inputting each test sample into a trained unbalanced hyperspectral image classification model, upsampling potential features corresponding to each sample by s times, and approximating a Bayes rule by using importance sampling to obtain the probability p (y | x) of a label to which each test sample belongs:
wherein softmax is a normalization function, L total As a function of total loss, x is the test sample, y c Is the label corresponding to the type c test sample,
is the potential feature corresponding to the class c test sample up-sampled by s times, theta is the trainable parameter of the three-dimensional encoder,
is the trainable parameter of the three-dimensional decoder and η is the trainable parameter of the classifier.
The effects of the present invention can be further explained by the following simulation results.
Simulation conditions
1. Test data: and (3) randomly selecting 1% of labeled samples as a training set and the rest samples as a testing set by using the University of Pavia data.
2. And (3) testing environment: a Ubuntu 16.04LTS system, NVIDIA GeForce GTX 1080Ti GPU and Tensorflow deep learning framework are adopted.
Second, simulation content
Simulation 1, under the above conditions, using the existing hyperspectral image classification method hyperspmoo based on small-class up-sampling of the generated countermeasure network to classify the University of Pavia data, and the result is shown in fig. 4.
By comparing fig. 4 with fig. 5, it can be found that the feature distribution of the classification result graph obtained by the method provided by the invention is closer to that of the truth graph, the classification effect obtained on most of small categories is better, and the better classification performance is realized.
Evaluation index
The two methods are tested 10 times by using the same 10 random numbers respectively, and the confusion matrix, the classification precision PA, the average classification precision AA, the overall classification precision OA and the KAPPA coefficient of each category are calculated, and the obtained test results are shown in Table 1.
The index calculation formula is as follows:
wherein TP is the number correctly divided into the present category, FN is the number of the present sample wrongly divided into other categories, TN is the number of the non-present sample divided into other categories, FP is the number of the other samples wrongly divided into the present sample,
x i+ represents the sum of all elements in the ith row in the confusion matrix, i.e. the sum of the actual number of this class, x +i Representing the sum of all elements in column i of the confusion matrix, i.e. the sum of the number of predicted cost classes, N tableThe sum of all elements is shown, c is the number of categories,
x ii the elements on the diagonal of the confusion matrix are represented, i.e. the correct number of predictions per category.
Table 1 simulation experiment results of the present invention and the comparison method
The bolded classes in table 1 are numbered as small sample classes, with the number of samples being less than the average number of samples per class.
As can be seen from table 1, compared to the current advanced hypergam method, the present invention improves the classification accuracy of the small categories with category numbers 3, 4, 5, 7, and 8, and also improves the average classification accuracy, the overall classification accuracy, and the KAPPA coefficient. In addition, the standard deviation obtained by the method is small, and the robustness of the method in the aspect of unbalanced hyperspectral classification is verified.
Claims (9)
1. A method for classifying unbalanced hyperspectral images is characterized by comprising the following steps of:
1) acquiring hyperspectral image data, selecting wave bands, segmenting the hyperspectral image after the wave band selection into spectrum-space characteristic neighborhood blocks according to a fixed size, and then dividing the spectrum-space characteristic neighborhood blocks according to the ratio of 1: 99 into training samples and test samples;
2) constructing an unbalanced hyperspectral image classification network model:
2a) establishing a three-dimensional encoder which is formed by three-dimensional convolution modules and two-dimensional convolution modules which are cascaded and are cascaded with two parallel fully-connected networks and then cascaded with a re-parameter module;
2b) setting a small-category up-sampling module in an implicit space;
2c) establishing a three-dimensional decoder consisting of 1 full-connection layer and a plurality of deconvolution layers;
2d) establishing an image classifier module consisting of 1 full connection layer;
2e) cascading a three-dimensional encoder and a small-category up-sampling module, connecting a three-dimensional decoder and a classifier in parallel, and simultaneously inputting the output of the small-category up-sampling module into the three-dimensional decoder and the classifier in parallel to form an unbalanced hyperspectral image classification network model;
3) training a non-equilibrium hyperspectral image classification network model:
3a) constructing the overall loss function L of the network model Total ＝λ·L mmd +L PdRec +L cls Wherein L is mmd For a regular term loss function based on maximum mean difference, L PdRec As a reconstruction loss function based on the neighborhood distance, L cls Is a classification loss function based on cross entropy, and lambda is a parameter;
3b) the parameters for initializing the unbalanced hyperspectral image classification network model comprise a weight parameter w, a bias parameter b and a parameter theta in a three-dimensional encoder, wherein the parameter theta is ═ theta 1 ,θ 2 Weight parameter w, bias parameter b and parameter in three-dimensional decoder
Mean value parameter in subclass sample characteristic up-sampling module
Sum variance parameter
And a classifier parameter η;
3c) inputting the training samples into the unbalanced hyperspectral image classification network model, circularly updating parameters in the unbalanced hyperspectral image classification network by adopting a gradient descent method to reduce the gradient value of the total loss function until the maximum iteration number T is reached, and obtaining the trained unbalanced hyperspectral image classification network model;
4) and inputting the test sample into a trained unbalanced hyperspectral image classification network model, and outputting a hyperspectral image classification result graph.
2. The method of claim 1, wherein:
acquiring hyperspectral image data and performing waveband selection in the step 1), namely performing waveband selection processing on the hyperspectral image by using a principal component analysis method to realize feature dimensionality reduction and reserving the first K principal components;
in the step 1), a spectrum-space characteristic neighborhood block is segmented from the hyperspectral image after the wave band selection according to a fixed size, wherein a primary component wave band is adopted to select samples, the size of each sample is S multiplied by K, and a spectrum-space neighborhood characteristic is obtained, wherein S represents the space size of a neighborhood;
the training image in 1) is represented as X ═ { X ═ X 1 ,...x i ...,x N Y ═ Y is the label corresponding to the training image 1 ,...y i ...,y N In which x i For the ith training image, y i For the ith label, i ranges from 1 to N, where N represents the number of training images.
3. The method of claim 1, wherein: the three-dimensional convolution modules in 2a) are all formed by cascading convolution layers, normalization layers and active layers, convolution kernels of the three convolution layers are (3,3,7), (3,3,5) and (3,3,3), the three normalization layers and the three active layers have the same structure, and the structure is as follows: the first convolution layer- > the first normalization layer- > the first active layer- > the second convolution layer- > the second normalization layer- > the second active layer- > the third convolution layer- > the third normalization layer- > the third active layer.
4. The method of claim 1, wherein: the two-dimensional convolution module in the 2a) is formed by cascading a convolution layer with convolution kernels (3,3), a normalization layer and an activation layer, and has the structure that: convolutional layer- > normalization layer- > activation layer.
5. The method of claim 1, wherein: the 2a) heavy parameter module is GaussianDistribution of
And (3) sampling a variable epsilon, and calculating the potential characteristics of the input image by combining the output of the full-connection layer: z is a radical of i Mu + epsilon sigma, where mu and sigma are the mean and variance parameters respectively output by the two parallel fully-connected networks,
x i for the ith input image, y i For labels corresponding to the input images, θ 2 Is a training parameter of the encoder.
6. The method of claim 1, wherein: and 2b) setting a small-class upsampling module in the implicit space, wherein the number of each sample of the small class in the implicit space is as follows:
wherein r represents an upsampling factor, n max Number of samples representing maximum class, n c Represents a sample x i And the number of the samples belonging to the class c realizes the expansion of the subclasses.
7. The method of claim 1, wherein: the three-dimensional decoder constructed in 2c) comprises a full connection layer and two deconvolution modules, each deconvolution module comprises a deconvolution layer, a normalization layer and an activation layer, convolution kernels of the two deconvolution layers are (3,3,7) and (3,3,5) respectively, and the two normalization layers and the two activation layers have the same structure;
the structure of the three-dimensional decoder is as follows:
the full-link layer- > the first deconvolution layer- > the first normalization layer- > the first activation layer- > the second deconvolution layer- > the second normalization layer- > the second activation layer.
8. The method of claim 1, wherein the step of removing the metal oxide is performed by a chemical vapor deposition processIn the following steps: the regularized term loss function L based on the maximum mean difference in 3a) mmd Reconstruction loss function L based on neighborhood distance PdRec Cross entropy based classification loss function L cls Respectively, as follows:
wherein p is θ (z i |x i ,y i ) A conditional probability distribution of an implicit characteristic is represented,
gaussian prior distribution, x, representing implicit characteristics i For the ith input image, y i For the label corresponding to the ith input image, z i For the ith input image, theta is a training parameter for the three-dimensional encoder,
training parameters of a three-dimensional decoder;
d represents the calculation of Euclidean distance, p ik Denotes x i Is detected in the image data of any one of the pixels,
represent
Is detected in the image data of any one of the pixels,
is x i Corresponding generated samples, S being image x i Side length of (y) ic And
x representing true and predicted respectively i Probability of belonging to class c.
9. The method of claim 1, wherein: the gradient descent method is adopted in the step 3c), parameters in the unbalanced hyperspectral image classification network are updated circularly, and the following steps are realized:
3c1) setting an initial value alpha of a learning rate, a maximum iteration number T and a batch processing number B, and initializing network training parameters;
3c2) classifying the training samples into batches according to the batch processing size, and sequentially inputting the batches into a non-equilibrium hyperspectral image classification networkUpdating total loss function by reverse propagation of line
According to a gradient of
And the descending direction carries out forward propagation to update the network training parameters. Performing the calculation on each batch of input networks until traversing the whole training set sample input network to finish one training;
3c3) and 3c2) is circulated until the maximum iteration time T is reached, and a trained unbalanced image classification model is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210626027.6A CN114882368A (en) | 2022-06-02 | 2022-06-02 | Non-equilibrium hyperspectral image classification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210626027.6A CN114882368A (en) | 2022-06-02 | 2022-06-02 | Non-equilibrium hyperspectral image classification method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114882368A true CN114882368A (en) | 2022-08-09 |
Family
ID=82680456
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210626027.6A Pending CN114882368A (en) | 2022-06-02 | 2022-06-02 | Non-equilibrium hyperspectral image classification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114882368A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115937703A (en) * | 2022-11-30 | 2023-04-07 | 南京林业大学 | Enhanced feature extraction method for remote sensing image target detection |
| CN117423004A (en) * | 2023-12-19 | 2024-01-19 | 深圳大学 | Band selection method, device, terminal and storage medium for hyperspectral image |
-
2022
- 2022-06-02 CN CN202210626027.6A patent/CN114882368A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115937703A (en) * | 2022-11-30 | 2023-04-07 | 南京林业大学 | Enhanced feature extraction method for remote sensing image target detection |
| CN115937703B (en) * | 2022-11-30 | 2024-05-03 | 南京林业大学 | Enhanced feature extraction method for remote sensing image target detection |
| CN117423004A (en) * | 2023-12-19 | 2024-01-19 | 深圳大学 | Band selection method, device, terminal and storage medium for hyperspectral image |
| CN117423004B (en) * | 2023-12-19 | 2024-04-02 | 深圳大学 | Band selection method, device, terminal and storage medium for hyperspectral image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111368896B (en) | Hyperspectral remote sensing image classification method based on dense residual three-dimensional convolutional neural network | |
| CN108960143B (en) | Ship detection deep learning method in high-resolution visible light remote sensing image | |
| CN109584248B (en) | Infrared target instance segmentation method based on feature fusion and dense connection network | |
| CN110363215B (en) | Method for converting SAR image into optical image based on generating type countermeasure network | |
| Xie et al. | Hyperspectral image super-resolution using deep feature matrix factorization | |
| Lin et al. | Hyperspectral image denoising via matrix factorization and deep prior regularization | |
| Zhao et al. | Fractional Fourier image transformer for multimodal remote sensing data classification | |
| CN113221641B (en) | Video pedestrian re-identification method based on generation of antagonism network and attention mechanism | |
| CN109684922B (en) | Multi-model finished dish identification method based on convolutional neural network | |
| CN110084159A (en) | Hyperspectral image classification method based on the multistage empty spectrum information CNN of joint | |
| CN114882368A (en) | Non-equilibrium hyperspectral image classification method | |
| CN113392931B (en) | Hyperspectral open set classification method based on self-supervision learning and multitask learning | |
| CN110544212B (en) | Convolutional neural network hyperspectral image sharpening method based on hierarchical feature fusion | |
| CN114241274B (en) | Small target detection method based on super-resolution multi-scale feature fusion | |
| CN112883887B (en) | Building instance automatic extraction method based on high spatial resolution optical remote sensing image | |
| Ge et al. | Adaptive hash attention and lower triangular network for hyperspectral image classification | |
| CN114863173B (en) | Self-mutual-attention hyperspectral image classification method for land resource audit | |
| CN116434069A (en) | Remote sensing image change detection method based on local-global transducer network | |
| CN114937173A (en) | Hyperspectral image rapid classification method based on dynamic graph convolution network | |
| CN112766223A (en) | Hyperspectral image target detection method based on sample mining and background reconstruction | |
| CN115311508A (en) | Single-frame image infrared dim target detection method based on depth U-type network | |
| Zhao et al. | GCFnet: Global collaborative fusion network for multispectral and panchromatic image classification | |
| CN116977747B (en) | Small sample hyperspectral classification method based on multipath multi-scale feature twin network | |
| CN115100509B (en) | Image identification method and system based on multi-branch block-level attention enhancement network | |
| CN116630700A (en) | Remote sensing image classification method based on introduction channel-space attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |