CN109858557B - Novel semi-supervised classification method for hyperspectral image data - Google Patents
Novel semi-supervised classification method for hyperspectral image data Download PDFInfo
- Publication number
- CN109858557B CN109858557B CN201910113156.3A CN201910113156A CN109858557B CN 109858557 B CN109858557 B CN 109858557B CN 201910113156 A CN201910113156 A CN 201910113156A CN 109858557 B CN109858557 B CN 109858557B
- Authority
- CN
- China
- Prior art keywords
- data
- hyperspectral
- classification
- training
- hyperspectral image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a novel semi-supervised classification algorithm of hyperspectral image data, which comprises the steps of firstly acquiring all hyperspectral data and taking the hyperspectral data as initial data; secondly, aggregating all initial data by using sparse subspace clustering as a clustering method, and acquiring pseudo labels of all initial data; secondly, pre-training all initial data and pseudo labels thereof in the CNN 1; for the objective function in the classification layer in the CNN1, a new objective function based on metric learning is proposed; then, CNN1 is used for removing the final classification layer, and a full connection layer is added to form CNN2; the tagged data and their known authentic tags are then trimmed to CNN 2. The invention relates to a semi-supervised deep learning mechanism for hyperspectral image classification by utilizing a pseudo label; two-stage training is used, and training information is expanded; in order to better master the problem of the structural characteristics of the hyperspectral sample, a new objective function is proposed in the pre-training stage.
Description
Technical Field
The invention belongs to the field of hyperspectral image classification, relates to a data semi-supervised classification technology, and particularly relates to a novel hyperspectral image data semi-supervised classification method.
Background
Through the development of the latter half of the 20 th century, the remote sensing technology has changed significantly in theory, technology and application. The hyperspectral image remote sensing technology is undoubtedly an important aspect in the great change of the remote sensing technology. The hyperspectral image remote sensing technology utilizes a hyperspectral sensor/imaging spectrometer to simultaneously image a target area by tens of to hundreds of continuously subdivided wave bands, combines an image with a spectrum, and simultaneously acquires spatial and spectral information to obtain a hyperspectral image with a pixel as a unit. The image contains a large amount of spectral information and is capable of collecting radiation or reflection information of the target in the ultraviolet, visible, near infrared and mid infrared regions of the electromagnetic spectrum.
The hyperspectral remote sensing data contains abundant spatial information and spectral information. The hyperspectral image has the main characteristic that continuous spectrums of ground objects and ground surface space images can be obtained simultaneously, and spectrum dimensional information and traditional space dimensional image information are fused together. The process of classifying the hyperspectral images is a process of classifying each pixel or a relatively uniform pixel group according to the spectral characteristic information and the spatial characteristic information of the image pixels.
Disclosure of Invention
The invention provides a semi-supervised classification algorithm for hyperspectral image data, which integrates spectral information and spatial information in a probability framework; firstly, acquiring pseudo labels of all data through sparse subspace clustering for pre-training, and adding a metric learning regularization term in a pre-training stage to construct a new objective function; then, fine adjustment is carried out by using the data with the label and the label thereof; the markov random field is then used as an image post-processing to further exploit the spatial information.
The invention aims to provide a novel semi-supervised classification method for hyperspectral image data.
The purpose of the invention can be realized by the following technical scheme:
a novel semi-supervised classification method for hyperspectral image data comprises the following steps:
the method comprises the following steps: acquiring all hyperspectral data, and taking the hyperspectral data as initial data;
step two: clustering all initial data by using sparse subspace clustering as a clustering method, and acquiring pseudo labels of all initial data; the pseudo label of the initial data is directly obtained through sparse subspace clustering;
step three: pre-training all initial data and pseudo labels thereof in the CNN 1;
for an objective function in a classification layer in the CNN1, a new objective function based on metric learning is provided;
step four: removing the final classification layer by using CNN1, and forming CNN2 by adding a full connection layer;
step five: fine-tuning the tagged data and their tags on CNN2;
step six: after the classification layer, the space information of the hyperspectral image is utilized through a Markov random field, the classification result is further subjected to post-processing, and the classification precision is improved.
Further, the initial data comprises marked hyperspectral data and unmarked hyperspectral data; and simultaneously training the CNN1 by using the marked hyperspectral data and the unmarked hyperspectral data.
Further, CNN1 in the third step refers to an initial deep convolutional neural network.
Further, the pre-training in the third step means that all initial data and pseudo labels thereof are put into a neural network CNN1 for training; pre-training can pre-train the network CNN1 through a large amount of initial data and obtain a set of appropriate parameters, and can learn the characteristics; since the marked data in the initial data is less, the unmarked hyperspectral data and the marked hyperspectral data are used.
Further, CNN2 in the fourth step refers to a new deep convolutional neural network; the method is formed by removing the last classification layer by CNN1 and then adding a full connection layer; the objective function for the classification layer in CNN2 is the softmax cross entropy loss function.
Further, the fine tuning in the fifth step means that the data with the real label and the label thereof are used as initial data and put into the CNN2 for training.
Further, the spatial information of the image is further utilized by the Markov random field in the sixth step, and the Markov random field encourages adjacent pixels to have the same class label in the image classification task; the classification accuracy of the hyperspectral image classification task can be further improved; the effect of the markov random field is to encourage neighboring pixels to belong to the same class of labels, assuming that the neighboring pixels are more likely to belong to the same class.
The invention has the beneficial effects that:
the invention relates to a semi-supervised deep learning mechanism for hyperspectral image classification by utilizing pseudo markers; two-stage training is used to extend the training information. Meanwhile, the invention provides a method for optimizing a new target loss function and adding a metric learning regularization condition term for characterizing hyperspectral data; and finally, the classification result is post-processed through a Markov random field, so that the spatial information of the hyperspectral image can be further utilized, the spatial information and the spectral information of the hyperspectral data are jointly extracted, namely, the space and the spectrum are combined, and the final classification effect is effectively improved.
Drawings
In order to facilitate understanding for those skilled in the art, the present invention will be further described with reference to the accompanying drawings.
FIG. 1 is a flow chart of a semi-supervised classification algorithm for hyperspectral image data according to the invention;
FIG. 2 is a block diagram of a semi-supervised deep learning framework of the present invention;
FIG. 3 is a block diagram of the pre-training and fine-tuning stages of the present invention.
Detailed Description
As shown in fig. 1-3, a novel semi-supervised classification method for hyperspectral image data comprises the following steps:
the method comprises the following steps: acquiring all hyperspectral data, and taking the hyperspectral data as initial data; the initial data comprises marked hyperspectral data and unmarked hyperspectral data;
step two: clustering all initial data by using sparse subspace clustering as a clustering method, and acquiring pseudo labels of all initial data;
wherein, the pseudo label of the initial data is directly obtained by sparse subspace clustering; the sparse subspace clustering algorithm has the main idea that any one sample can be represented by other sample points in an infinite number, so that the samples from the same subspace can be represented mutually, and only have representation coefficients, which means that in the infinite number of representations, most coefficients are 0, and are irrelevant samples, sparse representation is generated;
step three: pre-training all initial data and pseudo labels thereof in the CNN 1; the pre-training is to put all initial data and pseudo labels thereof into the neural network CNN1 for training.
Pre-training can pre-train the network CNN1 through a large amount of data and obtain a set of appropriate parameters, which can learn features; because the labeled data in the hyperspectrum is less, the large amount of data adopted by the pre-training also uses the unlabeled data;
for an objective function in a classification layer in the CNN1, a new objective function based on metric learning is provided; wherein the new objective function is referred to as J in FIG. 3; CNN1 refers to the initial deep convolutional neural network;
step four: removing the final classification layer by using the CNN1, and forming a CNN2 by adding a full connection layer; CNN2 refers to a new deep convolutional neural network; it is formed by removing the last classification layer by CNN1 and then adding a full connection layer. For the objective function of the classification layer in CNN2, it is a softmax cross entropy loss function.
Step five: fine-tuning the tagged data and their tags at CNN2; the fine tuning means that the data with the real label and the label thereof are used as initial data and put into the CNN2 for training.
Step six: after the hyperspectral image is subjected to the classification step, the spatial information of the hyperspectral image is further utilized by a Markov random field.
The specific principle of the invention can be explained as that the process of the hyperspectral image classification is a process of classifying each pixel or a relatively uniform pixel group according to the spectral characteristic information and the spatial characteristic information of the image pixels;
firstly, because the hyperspectral data are marked by few samples and only a small amount of marked data are used for training, sparse subspace clustering is used as a clustering method to aggregate all initial data and obtain pseudo labels of all data, all the initial data comprise unmarked hyperspectral data and marked hyperspectral data, and sparse subspace clustering is an excellent clustering method based on sparse expression and has good application effect in many fields;
next, all the initial data and the pseudo labels thereof are used for pre-training the deep convolutional neural network; unlike the traditional CNN model, which minimizes only cross-entropy loss, our proposed model is trained by optimizing a new discriminative objective function; for this reason, besides minimizing the softmax cross entropy loss function, we also add a metric learning regularization term; the metric learning regularization term is to utilize the contrast loss in metric learning and then combine the contrast loss with the cross entropy to construct a new target function in the pre-training stage CNN 1; and (3) metric learning, wherein the learning object is the distance of a sample feature vector generally, and the purpose of metric learning is to reduce or limit the distance between samples of the same type and simultaneously increase the distance between samples of different types through training and learning. Because there is a phenomenon in the hyperspectral image that different objects have the same spectrum and the same object has different spectra. Intra-class diversity and inter-class similarity remain two major challenges that generally degrade the performance of hyperspectral image classification. By adding the metric learning regularization term, the cluster can be gathered and the cluster can be dispersed.
Then, deleting the last layer from the pre-trained network, and adding a fully-connected layer and a softmax classification layer to construct a new deep network CNN2; at this time, we only use the tag data with the real class tag to fine tune the new deep network CNN2;
finally, a markov random field is used for further utilizing spatial information after the hyperspectral image classification step; in the image classification task, the Markov random field encourages adjacent pixels to have the same class label; this has been shown to greatly improve the classification accuracy of hyperspectral image classification tasks; the effect of the markov random field is to encourage neighboring pixels to belong to the same class of labels, assuming that they are more likely to belong to the same class.
The method performs semi-supervised deep learning of hyperspectral image classification by using the pseudo-markers; two-stage training is used to extend the training information. Meanwhile, the invention provides a metric learning regularization item which definitely emphasizes CNN characteristics by optimizing a new target loss function, and can better utilize spatial information and realize the combined extraction of spatial spectrum information in hyperspectral classification by integrating a Markov random field and CNN.
The foregoing is merely exemplary and illustrative of the present invention and various modifications, additions and substitutions may be made by those skilled in the art to the specific embodiments described without departing from the scope of the invention as defined in the following claims.
Claims (7)
1. A novel hyperspectral image data semi-supervised classification method is characterized by comprising the following steps of:
the method comprises the following steps: acquiring all hyperspectral data, and taking the hyperspectral data as initial data;
step two: clustering all initial data by using sparse subspace clustering as a clustering method, and acquiring pseudo labels of all initial data; the pseudo label of the initial data is directly obtained through sparse subspace clustering;
step three: pre-training all initial data and pseudo labels thereof in the CNN 1;
for the objective function in the classification layer in the CNN1, a new objective function based on metric learning is proposed;
step four: removing the final classification layer by using the CNN1, and forming a CNN2 by adding a full connection layer;
step five: fine-tuning the tagged data and their tags on CNN2;
step six: after the classification layer, the space information of the hyperspectral image is utilized through a Markov random field, the classification result is further subjected to post-processing, and the classification precision is improved.
2. The new semi-supervised classification method for hyperspectral image data according to claim 1, wherein the initial data comprises marked hyperspectral data and unmarked hyperspectral data; and simultaneously, training the CNN1 by using the marked hyperspectral data and the unmarked hyperspectral data.
3. The novel semi-supervised classification method for hyperspectral image data according to claim 1, wherein CNN1 in the third step refers to an initial deep convolutional neural network.
4. The novel semi-supervised classification method for hyperspectral image data according to claim 1 is characterized in that the pre-training in the third step is to put all initial data and pseudo labels thereof into a neural network CNN1 for training; pre-training can pre-train the network CNN1 through a large amount of initial data and obtain a set of appropriate parameters, and can learn the characteristics; since the marked data in the initial data is less, the unmarked hyperspectral data and the marked hyperspectral data are used.
5. The new semi-supervised classification method for hyperspectral image data according to claim 1, wherein CNN2 in the fourth step refers to a new deep convolutional neural network; the method is formed by removing the last classification layer by CNN1 and then adding a full connection layer; the objective function for the classification layer in CNN2 is the softmax cross entropy loss function.
6. The novel semi-supervised classification method for hyperspectral image data according to claim 1, wherein the fine tuning in the fifth step is to place data with real labels and labels thereof as initial data into CNN2 for training.
7. The method for semi-supervised classification of hyperspectral image data according to claim 1, wherein the spatial information of the hyperspectral image data is further utilized by the Markov random field in the sixth step, and the Markov random field encourages adjacent pixels to have the same class label in the image classification task; the classification accuracy of the hyperspectral image classification task can be further improved; the effect of the markov random field is to encourage neighboring pixels to belong to the same class of labels, assuming that they are more likely to belong to the same class.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910113156.3A CN109858557B (en) | 2019-02-13 | 2019-02-13 | Novel semi-supervised classification method for hyperspectral image data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910113156.3A CN109858557B (en) | 2019-02-13 | 2019-02-13 | Novel semi-supervised classification method for hyperspectral image data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109858557A CN109858557A (en) | 2019-06-07 |
| CN109858557B true CN109858557B (en) | 2022-12-27 |
Family
ID=66897911
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910113156.3A Active CN109858557B (en) | 2019-02-13 | 2019-02-13 | Novel semi-supervised classification method for hyperspectral image data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109858557B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309868A (en) * | 2019-06-24 | 2019-10-08 | 西北工业大学 | In conjunction with the hyperspectral image classification method of unsupervised learning |
| CN111401426B (en) * | 2020-03-11 | 2022-04-08 | 西北工业大学 | Small sample hyperspectral image classification method based on pseudo label learning |
| CN112541904B (en) * | 2020-12-16 | 2023-03-24 | 西安电子科技大学 | Unsupervised remote sensing image change detection method, storage medium and computing device |
| CN112784611A (en) * | 2021-01-21 | 2021-05-11 | 阿里巴巴集团控股有限公司 | Data processing method, device and computer storage medium |
| US20220309651A1 (en) * | 2021-03-24 | 2022-09-29 | Ping An Technology (Shenzhen) Co., Ltd. | Method, device, and storage medium for semi-supervised learning for bone mineral density estimation in hip x-ray images |
| CN113361628B (en) * | 2021-06-24 | 2023-04-14 | 海南电网有限责任公司电力科学研究院 | CNN insulator aging spectrum classification method under multi-task learning |
| CN115546567B (en) * | 2022-12-01 | 2023-04-28 | 成都考拉悠然科技有限公司 | Unsupervised domain adaptive classification method, system, equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017158058A1 (en) * | 2016-03-15 | 2017-09-21 | Imra Europe Sas | Method for classification of unique/rare cases by reinforcement learning in neural networks |
| CN107958216A (en) * | 2017-11-27 | 2018-04-24 | 沈阳航空航天大学 | Based on semi-supervised multi-modal deep learning sorting technique |
-
2019
- 2019-02-13 CN CN201910113156.3A patent/CN109858557B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017158058A1 (en) * | 2016-03-15 | 2017-09-21 | Imra Europe Sas | Method for classification of unique/rare cases by reinforcement learning in neural networks |
| CN107958216A (en) * | 2017-11-27 | 2018-04-24 | 沈阳航空航天大学 | Based on semi-supervised multi-modal deep learning sorting technique |
Non-Patent Citations (1)
| Title |
|---|
| 基于卷积神经网络的半监督高光谱图像分类;李绣心等;《电子测量与仪器学报》;20181015(第10期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109858557A (en) | 2019-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109858557B (en) | Novel semi-supervised classification method for hyperspectral image data | |
| Deng et al. | Deep metric learning-based feature embedding for hyperspectral image classification | |
| CN111695467B (en) | Spatial spectrum full convolution hyperspectral image classification method based on super-pixel sample expansion | |
| CN111652038B (en) | Remote sensing sea ice image classification method based on convolutional neural network | |
| CN109784392B (en) | Hyperspectral image semi-supervised classification method based on comprehensive confidence | |
| CN108460391B (en) | Hyperspectral image unsupervised feature extraction method based on generation countermeasure network | |
| CN111414942A (en) | Remote sensing image classification method based on active learning and convolutional neural network | |
| CN110728192A (en) | High-resolution remote sensing image classification method based on novel characteristic pyramid depth network | |
| He et al. | A dual global–local attention network for hyperspectral band selection | |
| CN107451565B (en) | Semi-supervised small sample deep learning image mode classification and identification method | |
| CN103955702A (en) | SAR image terrain classification method based on depth RBF network | |
| CN104616032A (en) | Multi-camera system target matching method based on deep-convolution neural network | |
| CN111639587B (en) | Hyperspectral image classification method based on multi-scale spectrum space convolution neural network | |
| CN111814685B (en) | Hyperspectral image classification method based on double-branch convolution self-encoder | |
| CN105184314B (en) | Wrapper formula EO-1 hyperion band selection methods based on pixel cluster | |
| CN110414616B (en) | Remote sensing image dictionary learning and classifying method utilizing spatial relationship | |
| Yang et al. | Local label descriptor for example based semantic image labeling | |
| CN114842264A (en) | Hyperspectral image classification method based on multi-scale spatial spectral feature joint learning | |
| He et al. | Multi-spectral remote sensing land-cover classification based on deep learning methods | |
| CN111639697B (en) | Hyperspectral image classification method based on non-repeated sampling and prototype network | |
| Qi et al. | Global-local three-dimensional convolutional transformer network for hyperspectral image classification | |
| CN114155443A (en) | Hyperspectral image classification method based on multi-receptive-field attention network | |
| CN112052758A (en) | Hyperspectral image classification method based on attention mechanism and recurrent neural network | |
| Guo et al. | CNN‐combined graph residual network with multilevel feature fusion for hyperspectral image classification | |
| CN115496934A (en) | Hyperspectral image classification method based on twin neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |