CN114494762A - A hyperspectral image classification method based on deep transfer network - Google Patents

Abstract

The invention discloses a hyperspectral image classification method based on a deep migration network. And the first-order statistic information of the related sub-fields is aligned based on the local maximum mean difference, and the local distribution of the two fields is adapted. The method can extract deep and discriminant features of the target domain, and only completes classification of the unlabeled samples of the target domain by using the source domain labeled samples.

Description

A hyperspectral image classification method based on deep transfer network

technical field

The invention relates to a method for realizing hyperspectral image classification by using a deep migration network, and belongs to the field of pattern recognition.

Background technique

Hyperspectral imagery (HSI) has rich spectral and spatial information, and it has broad application prospects in the fields of agriculture, climate detection, national defense and security. HSI classification is a common task of these applications, and its purpose is to classify each pixel in the image into different categories based on the spectral and spatial information obtained from the detected objects. Researchers have proposed many methods to improve the accuracy of HSI classification, including random forests, support vector machines, and decision trees. Although these traditional hyperspectral image classification methods have simple models, most of them cannot guarantee the classification accuracy.

In recent years, deep learning has achieved excellent performance on many computer vision tasks, such as image processing, object detection, and natural language processing. Deep learning can learn features automatically, which makes it applicable to various task scenarios. And deep learning has a powerful nonlinear representation ability, which can extract deeper and discriminative features of data. The above advantages enable deep learning to be successfully applied to hyperspectral image classification tasks. The powerful feature expression ability of deep learning often requires a large number of labeled training samples to support. As we all know, with the development of a new generation of satellite hyperspectral sensors, we can quickly acquire a large number of unlabeled HSIs, but labeling these images still requires a lot of time for relevant experts. Therefore, the lack of labeled samples seriously affects the application of deep learning methods on hyperspectral classification tasks. In order to solve the above problems, many researchers have completed hyperspectral image classification with a small number of labeled samples by combining active learning, data enhancement and other methods with deep learning. Although the above methods can alleviate the problem of insufficient training samples due to the difficulty of HSI labeling to a certain extent, when the training set and test set data come from similar but different fields, that is, the data distribution of the training set and test set is similar but When they are not the same, the above methods are difficult to obtain ideal experimental results. Transfer learning can solve this problem well. It transfers knowledge from a labeled domain (source domain) to a similar but different distribution domain (target domain) by exploring the domain-invariant structure, and then completes cross-domain classification. The deep transfer learning model obtained by combining deep network and transfer learning can learn deeper and more transferable features, and it has achieved breakthrough results in hyperspectral classification tasks. This stems from the fact that the deep transfer learning network can extract discriminative features and invariant factors in the data, and effectively group HSI features according to the correlation of the invariant factors.

SUMMARY OF THE INVENTION

Purpose of the invention: In order to overcome the deficiencies in the prior art, the present invention proposes a hyperspectral image classification method based on a deep transfer network, which can only use the labeled samples in the source domain to complete the classification of unlabeled samples in the target domain.

Technical solution: a hyperspectral image classification method based on a deep transfer network, comprising the following steps:

和目标域数据

Step 1, use band selection to reduce the dimension of the original hyperspectral image: remove the band redundancy to obtain the hyperspectral data X ₀ after dimension reduction, and the hyperspectral data X ₀ includes the source domain data after dimension reduction

and target domain data

Step 2, use the source domain labeled samples to train the auxiliary classifier, and use the auxiliary classifier to obtain the target domain pseudo-label;

Step 3, construct a deep transfer neural network, use the domain adaptation layer to adapt the source domain and the target domain based on the CORAL loss to reduce the second-order statistical difference, and at the same time reduce the first-order statistical difference between the source domain and the target correlation subspace based on LMMD;

Step 4, use the trained auxiliary classifier to classify the target domain data.

Further, in step 1, the original hyperspectral image is dimensionally reduced by band selection: the band redundancy is removed to obtain the dimension-reduced hyperspectral data X ₀ , which specifically includes the following processes:

和

分别选择a和b个波段，降维后的波段数为d，其中

表示向下取整运算，可得：Define the number of original HSI bands as N _b , with the number of intervals as

and

Select a and b bands respectively, and the number of bands after dimension reduction is d, where

Represents a round-down operation, which results in:

和目标域数据

其中，

和

其中n_s表示源域样本数，n_t表示目标域样本数，Y^s为源域标签。Define the dimension-reduced hyperspectral data X ₀ ∈ R ^n×d as the input of the model, n represents the number of samples, and the spectral data X ₀ includes the dimension-reduced source domain data

and target domain data

in,

and

where _ns represents the number of samples in the source domain, _nt represents the number of samples in the target domain, and Y ^s is the label in the source domain.

Further, in step 3, a deep transfer neural network is constructed, and the domain adaptation layer is used to adapt the source domain and the target domain based on the CORAL method to reduce the difference of the second-order statistics, and at the same time, the first-order statistics of the source domain and the target correlation subspace are reduced based on LMMD. Quantitative differences; specifically include the following:

和目标域数据

输入深度迁移网络DTN中，经由全连接层和非线性层构成的特征提取器提取特征；其中，全连接层输入为：The deep transfer network DTN includes a fully connected layer, a nonlinear layer, a domain adaptation layer and a Softmax layer; the source domain data after dimension reduction

and target domain data

In the input deep transfer network DTN, features are extracted through a feature extractor composed of a fully connected layer and a nonlinear layer; wherein, the input of the fully connected layer is:

F ₁ =I×W ₁ +b ₁

Among them, I is the input of the fully connected layer, and b ₁ is the bias; the output of the fully connected layer is connected as the input to the nonlinear layer, and the output of the nonlinear layer is:

Among them, I ^N is the nonlinear layer input;

The domain adaptation layer is used to adapt the distribution difference between the two domains, and then the output of the domain adaptation layer is connected to the Softmax layer; the loss function of the deep transfer network DTN is defined as:

为协方差领域适配项，

为子空间适应项，

为源域数据分类损失，α₁和α₂分别为方差领域适应参数和子空间适应参数；协方差领域适配项可表示为：in,

is the adaptation term for the covariance field,

is the subspace adaptation term,

is the source domain data classification loss, α ₁ and α ₂ are the variance domain adaptation parameters and subspace adaptation parameters, respectively; the covariance domain adaptation term can be expressed as:

Among them, d ₁ is the input dimension of the domain adaptation layer, and C _s and C _t represent the source domain and target domain data covariance matrices, respectively;

The subspace adaptation term can be expressed as:

为经辅助分类器得到的目标域伪标签，c∈{1,2,...,C}为类别索引，

和

表示

和

属于c类的权重，则

是c类的加权和，

可计算为：in,

is the pseudo-label of the target domain obtained by the auxiliary classifier, c∈{1,2,...,C} is the category index,

and

express

and

belongs to the weight of class c, then

is the weighted sum of class c,

can be calculated as:

The source domain data classification loss can be expressed as:

Among them, C is the number of categories, Y is the category matrix, and S is the prediction result of the deep transfer network DTN model.

Beneficial effects: The cross-domain hyperspectral image classification method of the deep transfer network of the present invention not only aligns the second-order statistics information of the two domains as a whole based on CORAL, but also adapts the global distribution of the two domains. And based on the local maximum mean difference, the first-order statistic information of the relevant sub-domains is aligned, and the local distribution of the two domains is adapted. And the proposed method can extract the deep and discriminative features of the target domain, and only use the labeled samples in the source domain to complete the classification of unlabeled samples in the target domain.

Description of drawings

Figure 1 is a schematic diagram of the method of the present invention.

Detailed ways

The present invention will be further explained below in conjunction with the accompanying drawings.

A hyperspectral image classification method based on a deep transfer network, which can complete the classification of unlabeled samples in the target domain using only the labeled samples in the source domain.

Technical solution: a hyperspectral image classification method based on a deep transfer network, comprising the following steps:

和目标域数据

Step 1, use band selection to reduce the dimension of the original hyperspectral image: remove the band redundancy to obtain the hyperspectral data X ₀ after dimension reduction, and the hyperspectral data X ₀ includes the source domain data after dimension reduction

and target domain data

和

分别选择a和b个波段，降维后的波段数为d，其中

表示向下取整运算，可得：The number of original HSI bands is large and the correlation between bands is strong, so there is a lot of redundant information between bands. Directly inputting the original HSI into the deep transfer network DTN will increase the network parameters and degrade the model performance. Apply band selection to dimensionality reduction of raw HSI data. Define the number of original HSI bands as N _b , with the number of intervals as

and

Select a and b bands respectively, and the number of bands after dimension reduction is d, where

Represents a round-down operation, which results in:

和目标域数据

其中，

和

其中n_s表示源域样本数，n_t表示目标域样本数，Y^s为源域标签。Define the dimension-reduced hyperspectral data X ₀ ∈ R ^n×d as the input of the model, n represents the number of samples, and the spectral data X ₀ includes the dimension-reduced source domain data

and target domain data

in,

and

where _ns represents the number of samples in the source domain, _nt represents the number of samples in the target domain, and Y ^s is the label in the source domain.

Step 2, use the source domain labeled samples to train the auxiliary classifier, and use the auxiliary classifier to obtain the target domain pseudo-label;

Step 3, build a deep transfer neural network, based on the CORAL (CORrelation Alignment) algorithm, use the domain adaptation layer to reduce the second-order statistical difference between the two domains, and reduce the source based on LMMD (Local Maximum Mean Discrepancy, Local Maximum Mean Measure). Differences in first-order statistics of domain and target-related subspaces.

Deep neural networks are widely used in HSI classification due to their powerful deep feature extraction capabilities. However, when the training set and the test set belong to different data distributions, it is difficult for the deep neural network to learn transferable knowledge, resulting in insufficient model classification ability. In order to solve the above problems, a deep transfer network DTN is proposed, and a domain adaptation layer is added to the DNN, and the deep and discriminative source and target domain features extracted by the DNN are simultaneously processed for global second-order statistics and each type of correlation subspace. First-order statistic alignment.

The deep transfer network DTN is a feedforward neural network including a fully connected layer, a nonlinear layer, a domain adaptation layer and a Softmax layer. FC1 in Figure 1 represents the domain adaptation layer, and FC2 represents the Softmax layer.

和目标域数据

输入深度迁移网络DTN中，经由全连接层和非线性层构成的特征提取器提取特征。其中，全连接层输入为：source domain data after dimensionality reduction

and target domain data

In the input deep transfer network DTN, features are extracted through a feature extractor composed of fully connected layers and nonlinear layers. Among them, the input of the fully connected layer is:

F ₁ =I×W ₁ +b ₁

Among them, I is the input of the fully connected layer, and b ₁ is the bias. The fully connected layer output is connected as input to the nonlinear layer, and the output of the nonlinear layer is:

Among them, I ^N is the nonlinear layer input. A domain adaptation layer is added to adapt to the distribution difference between the two domains, and then the output of the domain adaptation layer is connected to the Softmax layer. The loss function of the deep transfer network DTN is defined as:

为协方差领域适配项，

为子空间适应项，

为源域数据分类损失，α₁和α₂分别为方差领域适应参数和子空间适应参数。in,

is the adaptation term for the covariance field,

is the subspace adaptation term,

is the classification _loss for the source domain data, and α1 and _α2 are the variance domain adaptation parameters and subspace adaptation parameters, respectively.

The covariance domain fit term can be expressed as:

Among them, d ₁ is the input dimension of the domain adaptation layer, and C _s and C _t represent the source domain and target domain data covariance matrices, respectively.

The subspace adaptation term can be expressed as:

为经辅助分类器得到的目标域伪标签，c∈{1,2,...,C}为类别索引，

和

表示

和

属于c类的权重，则

是c类的加权和，

可计算为：in,

is the pseudo-label of the target domain obtained by the auxiliary classifier, c∈{1,2,...,C} is the category index,

and

express

and

belongs to the weight of class c, then

is the weighted sum of class c,

can be calculated as:

The source domain data classification loss can be expressed as:

Among them, C is the number of categories, Y is the category matrix, and S is the prediction result of the deep transfer network DTN model.

Step 4: Classify the target domain data using the trained auxiliary classifier.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims (3)

1. a hyperspectral image classification method based on deep migration network, is characterized in that, comprises the steps: Step 1, use band selection to reduce the dimension of the original hyperspectral image: remove the band redundancy to obtain the hyperspectral data X ₀ after dimension reduction, and the hyperspectral data X ₀ includes the source domain data after dimension reduction

and target domain data

Step 2, use the source domain labeled samples to train the auxiliary classifier, and use the auxiliary classifier to obtain the target domain pseudo-label; Step 3, construct a deep transfer neural network, use the domain adaptation layer to adapt the source domain and the target domain based on the CORAL loss to reduce the second-order statistical difference, and at the same time reduce the first-order statistical difference between the source domain and the target correlation subspace based on LMMD; Step 4, use the trained auxiliary classifier to classify the target domain data. 2. A kind of hyperspectral image classification method based on deep migration network according to claim 1, it is characterized in that, in step 1, use band selection to carry out dimension reduction to original hyperspectral image: remove band redundancy, obtain after dimensionality reduction. The hyperspectral data X ₀ specifically includes the following processes: Define the number of original HSI bands as N _b , with the number of intervals as

and

Select a and b bands respectively, and the number of bands after dimension reduction is d, where

Represents a round-down operation, which results in:

Define the dimension-reduced hyperspectral data X ₀ ∈ R ^n×d as the input of the model, n represents the number of samples, and the spectral data X ₀ includes the dimension-reduced source domain data

and target domain data

in,

and

where _ns represents the number of samples in the source domain, _nt represents the number of samples in the target domain, and Y ^s is the label in the source domain. 3. a kind of hyperspectral image classification method based on deep transfer network according to claim 1, it is characterized in that, build deep transfer neural network in step 3, utilize domain adaptation layer to adapt source domain and target domain subtraction based on CORAL method. Small second-order statistic difference, while reducing the first-order statistic difference between the source domain and the target correlation subspace based on LMMD; the details include the following: The deep transfer network DTN includes a fully connected layer, a nonlinear layer, a domain adaptation layer and a Softmax layer; the source domain data after dimension reduction

and target domain data

In the input deep transfer network DTN, features are extracted through a feature extractor composed of a fully connected layer and a nonlinear layer; wherein, the input of the fully connected layer is: F ₁ =I×W ₁ +b ₁ Among them, I is the input of the fully connected layer, and b ₁ is the bias; the output of the fully connected layer is connected as the input to the nonlinear layer, and the output of the nonlinear layer is:

Among them, I ^N is the nonlinear layer input; The domain adaptation layer is used to adapt the distribution difference between the two domains, and then the output of the domain adaptation layer is connected to the Softmax layer; the loss function of the deep transfer network DTN is defined as:

in,

is the adaptation term for the covariance field,

is the subspace adaptation term,

is the source domain data classification loss, α ₁ and α ₂ are the variance domain adaptation parameters and subspace adaptation parameters, respectively; the covariance domain adaptation term can be expressed as:

Among them, d ₁ is the input dimension of the domain adaptation layer, and C _s and C _t represent the source domain and target domain data covariance matrices, respectively; The subspace adaptation term can be expressed as:

in,

is the pseudo-label of the target domain obtained by the auxiliary classifier, c∈{1,2,...,C} is the category index,

and

express

and

belongs to the weight of class c, then

is the weighted sum of class c,

can be calculated as:

The source domain data classification loss can be expressed as:

Among them, C is the number of categories, Y is the category matrix, and S is the prediction result of the deep transfer network DTN model.

Images