CN116977744A - Small sample cross-domain hyperspectral image classification method based on contrast learning and subspace - Google Patents
Small sample cross-domain hyperspectral image classification method based on contrast learning and subspace Download PDFInfo
- Publication number
- CN116977744A CN116977744A CN202310979029.8A CN202310979029A CN116977744A CN 116977744 A CN116977744 A CN 116977744A CN 202310979029 A CN202310979029 A CN 202310979029A CN 116977744 A CN116977744 A CN 116977744A
- Authority
- CN
- China
- Prior art keywords
- domain
- source domain
- subspace
- feature
- target domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000012549 training Methods 0.000 claims abstract description 55
- 230000006870 function Effects 0.000 claims abstract description 44
- 238000012360 testing method Methods 0.000 claims abstract description 33
- 238000013507 mapping Methods 0.000 claims abstract description 27
- 230000004927 fusion Effects 0.000 claims description 39
- 238000011176 pooling Methods 0.000 claims description 17
- 239000013598 vector Substances 0.000 claims description 15
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 11
- 238000010606 normalization Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 7
- OQCFWECOQNPQCG-UHFFFAOYSA-N 1,3,4,8-tetrahydropyrimido[4,5-c]oxazin-7-one Chemical compound C1CONC2=C1C=NC(=O)N2 OQCFWECOQNPQCG-UHFFFAOYSA-N 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000000354 decomposition reaction Methods 0.000 claims description 5
- 238000009825 accumulation Methods 0.000 claims description 4
- 238000002474 experimental method Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 238000005259 measurement Methods 0.000 claims 1
- 238000012544 monitoring process Methods 0.000 abstract description 3
- 238000007796 conventional method Methods 0.000 abstract description 2
- 238000004088 simulation Methods 0.000 description 10
- 241001466077 Salina Species 0.000 description 6
- 230000002776 aggregation Effects 0.000 description 4
- 238000004220 aggregation Methods 0.000 description 4
- 230000003595 spectral effect Effects 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 239000010426 asphalt Substances 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000013526 transfer learning Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 240000007087 Apium graveolens Species 0.000 description 1
- 235000015849 Apium graveolens Dulce Group Nutrition 0.000 description 1
- 235000010591 Appio Nutrition 0.000 description 1
- 241000287196 Asthenes Species 0.000 description 1
- 244000075850 Avena orientalis Species 0.000 description 1
- 235000007319 Avena orientalis Nutrition 0.000 description 1
- 240000004658 Medicago sativa Species 0.000 description 1
- 235000017587 Medicago sativa ssp. sativa Nutrition 0.000 description 1
- 241000209140 Triticum Species 0.000 description 1
- 235000021307 Triticum Nutrition 0.000 description 1
- 241001532014 Xanthorrhoea Species 0.000 description 1
- 240000008042 Zea mays Species 0.000 description 1
- 235000005824 Zea mays ssp. parviglumis Nutrition 0.000 description 1
- 235000002017 Zea mays subsp mays Nutrition 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 239000011449 brick Substances 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 235000005822 corn Nutrition 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- ZZUFCTLCJUWOSV-UHFFFAOYSA-N furosemide Chemical compound C1=C(Cl)C(S(=O)(=O)N)=CC(C(O)=O)=C1NCC1=CC=CO1 ZZUFCTLCJUWOSV-UHFFFAOYSA-N 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0475—Generative networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/094—Adversarial learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/58—Extraction of image or video features relating to hyperspectral data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V10/7753—Incorporation of unlabelled data, e.g. multiple instance learning [MIL]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/776—Validation; Performance evaluation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Multimedia (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Medical Informatics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Biodiversity & Conservation Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a small sample cross-domain hyperspectral image classification method based on contrast learning and subspace, which mainly solves the problems of low utilization rate of embedded characteristic information, high training cost and low precision of a conventional method. The scheme comprises the following steps: acquiring a hyperspectral data set, and dividing a source domain and a target domain to acquire a training set and a testing set; constructing a mapping layer and a feature extractor, and respectively extracting embedded features of a source domain and a target domain by using the mapping layer and the feature extractor to perform subspace classification and contrast learning; constructing a domain discriminator, and calculating the total loss of domain discrimination loss, space loss and contrast loss of a source domain and a target domain; performing iterative training on the feature extractor by using the training set through back propagation until the total loss function converges; and inputting the test set into a trained feature extractor to obtain a classification result. The invention improves the distinguishing capability and the classifying precision of the classifier, improves the utilization rate of embedded characteristic information, reduces the training cost, and can be used for land feature classification of resource exploration, forest coverage and disaster monitoring.
Description
Technical Field
The invention belongs to the technical field of image processing, and further relates to a small sample cross-domain hyperspectral image classification method which can be applied to ground object classification and target identification in resource exploration, forest covering and disaster monitoring.
Background
Compared with the traditional image, the hyperspectral image HSI contains a plurality of continuous wave bands, contains a large amount of spectrum and space information, and is widely applied to the fields of environment monitoring, disaster prevention and the like due to the fact that the hyperspectral image HSI is rich in information. In practical remote sensing applications, there is often a limited number of marked samples available for training, a challenge called small sample classification task. In the early days, many machine learning methods were developed for small sample scene classification, such as principal component analysis, filtering, etc., but the classification effect was not ideal. Then, with the development of deep learning, various neural networks have made great progress in the field of classification of hyperspectral images of large samples, but have still progressed slowly in the field of small samples. In recent years, in order to solve the problem of small samples, many methods such as transfer learning, active learning, contrast learning, meta learning, and the like have appeared. Specialized small sample learning, such as prototype networks, twinning networks, relational networks, etc., have also been developed. Today, researchers are beginning to focus on how to extend small sample learning into a cross-domain environment, solving hyperspectral image classification where the source domain dataset is different from the target domain data.
Gao et al in its published paper "Deep relation network for hyperspectral image few-shot classification, remote sens, vol.12, no.6, p.923, mar.2020" propose a small sample classification method based on a relational network. The method comprises three steps: firstly, performing meta-learning on a source domain HSI data set by using a designed feature learning module and a relation learning module; secondly, fine tuning the network model by using a small number of labeled samples in the target domain data set; third, the classification performance is tested using the target HSI dataset. The method uses a fine tuning means to adjust domain offset between a source domain and a target domain, so that the efficiency is low, the stability and the robustness of a network are not facilitated, and meanwhile, a relationship network adopted by the method has a great room for improvement.
Li et al in its published paper "Deep cross-domain few-shot learning for hyperspectral image classification, IEEE Trans. Geosci. Remote Sens., vol.60, pp.1-18,2021," propose to overcome domain bias using a conditional anti-domain adaptation strategy to achieve domain distribution alignment. Firstly, a unified framework is established, and source domain data and target domain data are mapped to the same dimension through a mapping layer; extracting embedded features by using the same feature extractor, and simultaneously carrying out small sample learning in a source domain and a target domain; and the source domain and target domain data are identified by using a conditional domain identifier, so that inter-domain offset is relieved, and cross-domain small sample classification is realized. The method adopts a small sample method based on a prototype network to classify, and the prototype uses the mean value of the embedded features, so that the utilization rate of the embedded feature information is limited, and higher classification precision cannot be achieved.
Zhang et al in its published paper "Graph information aggregation cross-domain new-shot learning for hyperspectral image classification IEEE Trans.Neal Net. Learn. Syst., early access, jun.30,2022" propose a graph information aggregation cross-domain few-sample learning framework. The implementation steps are as follows: firstly, dividing a training set and a testing set on a source domain and a target domain, and mapping to the same dimension through a mapping layer; secondly, after extracting embedded features of a source domain and a target domain, learning the similarity between a support set and a query set by using a metric function; third, the ided blocks and csa blocks of the source domain and the target domain are designed, and domain alignment is performed from two layers of graph characteristics and graph distribution. The method has the defects that the embedded characteristics of the source domain and the target domain are processed too much, so that the training cost is high, and the utilization of other information is limited.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a small sample cross-domain hyperspectral image classification method based on contrast learning and subspace, so as to improve the utilization rate of embedded characteristic information, reduce training cost and further improve classification precision.
The technical idea for realizing the purpose of the invention is as follows: classifying the embedded features of the source domain and the target domain by using a subspace improved prototype network, so that the utilization rate of the embedded feature information is improved; constructing an integral model for connecting a source domain and a target domain through a domain discriminator, realizing domain alignment, and simultaneously reducing training cost; by introducing contrast learning to analyze the correlation and the difference between the source domain samples and combining subspace and domain discrimination, the classification precision of the cross-domain small samples is further improved.
According to the above idea, the implementation steps of the invention include the following:
(1) Acquiring a hyperspectral data set and dividing:
(1a) G hyperspectral data sets are obtained from a public website, one data set is used as a source domain, and the remaining G-1 data sets are used as target domains;
(1b) C categories are selected from the source domain data set, and K source domain samples are selected as a source domain support set S from each category s Then N other source domain samples are selected to form a source domain query set Q s ；
(1c) H tagged categories are selected from the target domain data set, and K target domain tagged samples are selected from each category as a target domain support set S t N other target domains are selected again by each class and have label samples to form a target domain query set Q t Support set S with target domain t Query set Q with target domain t Together form a training set T r The method comprises the steps of carrying out a first treatment on the surface of the Taking the rest target domain unlabeled sample as a test set T e ；
(2) Constructing a mapping layer comprising a two-dimensional convolution layer and a BatchNorm2d layer; constructing a feature extractor f comprising two residual blocks, two max-pooling layers, one convolutional layer and one flame layer θ ；
(3) Subspace classification is carried out on embedded features of a source domain and a target domain:
(3a) Support set S of source domain s Source domain query set Q s Target domain support set S t Target domain query set Q t Respectively input to the mapping layer and the feature extractor f θ Obtaining the embedding characteristic f of the source domain support set θ (S s ) Embedding feature f in source domain query set θ (Q s ) Target domain support set embedding feature f θ (S t ) Target domain query set embedding feature f θ (Q t )；
(3b) Embedding feature f with source domain support set θ (S s ) Constructing a source domain subspace and calculating a source domain query set embedding feature f θ (Q s ) Source domain subspace loss in source domain subspace/ S ；
(3c) Computing source domain support set embedding features f θ (S s ) Source domain support loss in source domain subspace
(3d) Embedding feature f with target domain support set θ (S t ) Constructing a target domain subspace and calculating an embedded feature f of a target domain query set θ (Q t ) Target domain subspace loss in target domain subspace/ T ；
(3e) Computing target domain support set embedding features f θ (S t ) Target domain support loss in target domain subspace
(4) Support set S of source domain s Dividing into C contrast groups, and calculating contrast loss l of the input mapping layer and the feature extractor CL ；
(5) Constructing a domain discriminator f comprising five full connection layers, four ReLU nonlinear activation functions, four dropout layers, a Softmax activation function D ；
(6) Calculating a domain discrimination penalty l D ：
(6a) Loss of source domain subspace/ S Loss of source domain supportSource domain support set embedding feature f θ (S s ) Embedding feature f in source domain query set θ (Q s ) Fusion is performed to obtain a source domain fusion feature (T (h s ))；
(6b) Loss of target domain subspace/ T Loss of target domain supportTarget domain support set embedding feature f θ (S t ) Target domain query set embedding feature f θ (Q t ) Fusion is performed to obtain a target domain fusion feature (T (h) t ))；
(6c) The fusion characteristics of the source domain and the target domain (T (h s ))、(T(h t ) Input field discriminator f) D Medium calculation domain discrimination loss l D ：
Wherein D, T respectively represent the fusion characteristics of the domain discriminator and the characteristic extractor, and x s ，x t Respectively represent source domain sample and destinationStandard domain sample, P s (x)，P t (x) Representing the source domain distribution and the target domain distribution, respectively, D (T (h s ) Representing a discriminator f D Prediction source domain fusion features (T (h) s ) A) the probability of originating from the source domain,representation discriminator f D Predicting target domain fusion features The probability from the target domain, E, is the expected output;
(7) Computing a total loss function to train the feature extractor:
calculating the subspace loss l of the source domain S Target domain subspace loss l T Loss of contrast l CL Domain discrimination loss l D Total loss function of composition: l=λ 1 l S +λ 2 l T +λ 3 l CL +l D, wherein λ1 、λ 2 、λ 3 Respectively is l S 、l T 、l CL Is provided;
feature extractor f by back propagation θ Iterative training is carried out until the total loss function converges, and a trained feature extractor f is obtained θ '；
(8) Classifying by using test set data:
(8a) Training set T r Input to the mapping layer and trained feature extractor f θ In' get training set embedding feature f θ '(T r ) By utilizing the characteristic f θ '(T r ) Constructing a training set subspace;
(8b) Test set T e Input to the mapping layer and trained feature extractor f θ In' get test set embedded feature f θ '(T e ) And calculating a test subspace loss l' thereof in the training set subspace;
(8c) And taking the smallest category in the test subspace loss l' as a prediction category to finish the classification task.
Compared with the prior art, the invention has the following advantages:
firstly, the invention adopts subspace classification calculation support set and query set to embed the loss of the characteristics, thus solving the problem that the conventional prototype network only takes the average value of the embedded characteristics as the prototype, resulting in the loss of the utilization rate of multidimensional data, improving the discrimination capability and generalization performance of the classifier and improving the utilization rate of the embedded characteristic information.
Secondly, the invention uses the domain discriminator to project the fusion characteristics of the source domain and the target domain into the public space, thereby overcoming the complex setting and adjustment of the conventional domain adaptation method and the migration method to the cross-domain data; meanwhile, as the cross-domain data is subjected to feature learning and representation learning in a generation-countermeasure mode, an integral model for connecting the source domain and the target domain is constructed to realize domain alignment, and training cost is reduced.
Thirdly, the invention uses contrast learning to train the source domain support set data, thereby overcoming the bottleneck of precision improvement caused by the similarity and the difference of neglecting the same type samples in the conventional method, and further improving the classification precision of the cross-domain small samples.
Drawings
FIG. 1 is a flow chart of an implementation of the present invention;
FIG. 2 is a schematic diagram of a feature extractor constructed in accordance with the present invention;
fig. 3 is a schematic diagram of a domain discriminator constructed in the invention.
Detailed Description
The invention will be described in further detail with reference to the drawings and the specific embodiments, it being understood that the specific embodiments are for illustration only and not for limitation of the invention.
Referring to fig. 1, the implementation steps for this example include the following:
step 1, carrying out cross-domain small sample division on a data set.
The cross-domain small sample classification refers to classification under the condition that fewer labeled samples are available for training and source domain data and target domain data are different.
The specific implementation of the steps is as follows:
g hyperspectral data sets are obtained from the public website, one data set is used as a source domain, and the remaining G-1 data sets are used as target domains;
for a source domain data set, C classes are randomly selected from the source domain data set, and K samples are randomly extracted from each class to serve as a source domain support set, which is called C-wayK-shot for short;
representing a source domain support set asThen N samples are selected from the C classes as a source domain query set +.>Wherein x represents a sample and y represents a label of the sample;
for the target domain data set, H tagged categories are selected from the target domain data set, and K target domain tagged samples are selected from each category as target domain support setsSelecting N other target domains with label samples to form target domain query set
Support set S with target domain t Query set Q with target domain t Together form a training set T r Taking the rest target domain unlabeled sample as a test set T e ；
This example selects, but is not limited to, c=19, k=2, n=19, g=4, h selects 16,9, 16, respectively, in the 3 target domain datasets.
And 2, constructing a mapping layer and a feature extractor.
2.1 Building a mapping layer:
2.1.1 The input of the two-dimensional convolution layer is selected as the input data dimension, the output is 100, the convolution kernel size is 1 multiplied by 1, and the step length is 1;
2.1.2 Selecting two-dimensional batch normalizationAs a BatchNorm2d layer, where x represents the input data, mean (x) represents the mean, var (x) represents the variance, eps is a minimum, γ, β are two different values of the adjustment parameters, in this example, γ=1, β=0;
2.1.3 A mapping layer formed by sequentially connecting a two-dimensional convolution layer and a BatchNorm2d layer so as to map a source domain and a target domain to the same dimension, so that the source domain and the target domain can be trained in the same feature extractor;
2.2 Constructing a feature extractor:
referring to fig. 2, the implementation of this step includes the following:
2.2.1 A convolution block consisting of a 3D convolution layer and a BatchNorm3D layer is selected, the convolution kernel size of the 3D convolution layer is 3 multiplied by 3, the step length is 1, and the filling is 1; the function of each ReLU layer is ReLU (x) =max (x, 0);
2.2.2 Two residual blocks with the same structure are selected, each residual block comprises 3 convolution blocks and 3 ReLU layers, and the structural relation is as follows: the first convolution block, the first ReLU layer, the second convolution block, the second ReLU layer, the third convolution block and the third ReLU layer, and the output of the first ReLU layer is connected with the third convolution block in a residual way;
2.2.3 The two largest pooling layers with different parameters are selected, wherein the pooling core size of the first pooling layer is 4 multiplied by 2, the filling is 0 multiplied by 1, and the step length is 4 multiplied by 2; the size of the pooling core of the second pooling layer is 4 multiplied by 2, the filling is 4 multiplied by 2, and the step size is 2 multiplied by 1;
2.2.4 A convolution layer with the output channel number of 32 and the convolution kernel size of 3 multiplied by 3 is selected;
2.2.5 A first residual block, a first pooling layer, a second residual block, a second pooling layer and a convolution layer are sequentially connected to form a feature extractor f θ The training device is used for training the input data to obtain the embedded characteristics of the input data.
And step 3, classifying subspaces of the embedded features of the source domain and the target domain.
Currently, common small sample learning classification methods include, for example, twin networks, relational networks, prototype networks, and the like; the twin network classifies by minimizing losses between embedded features of the same class and maximizing losses between embedded features of different classes; the relation network analyzes the matching degree by calculating the distance between the embedded features of the two samples to classify; the prototype network classifies the embedded features of the query set by calculating the distance from the embedded features of the support set to the average value of the embedded features of the support set; the method adopts, but is not limited to, a subspace improvement prototype network, and classifies the query set embedded features by calculating subspace distances from the query set embedded features to the support set embedded features, so that the discrimination capability and generalization performance of the classifier are enhanced, and the utilization rate of the embedded feature information is improved. The implementation steps comprise the following steps:
3.1 Source domain support set S) s Source domain query set Q s Target domain support set S t Target domain query set Q t Respectively inputting to a mapping layer and a feature extractor to obtain a source domain support set embedded feature f θ (S s ) Embedding feature f in source domain query set θ (Q s ) Target domain support set embedding feature f θ (S t ) Target domain query set embedding feature f θ (Q t )；
3.2 Embedding features f using source domain support sets θ (S s ) Constructing a source domain subspace and calculating a source domain query set embedding feature f θ (Q s ) Source domain subspace loss in source domain subspace/ S ：
3.2.1 Embedding features f) into a source domain support set θ (S s ) Singular value decomposition f θ (X c )＝B c Σ c V c T, wherein Xc Representing a source domain support set S s Class c, B c Called left singular vectors, V c T Called right singular vectors, T represents the transpose, Σ c Is a singular value matrix;
3.2.2 Selecting left singular vector B) c Form a truncated matrix P from the first n dimensions of (1) c And P is taken c A base regarded as a source domain subspace; in this example
3.2.3 Computing a source domain query set embedding feature f θ (Q s ) Sample q of (3) k In the source domain subspace P c Projection distance d of (2) c (q k )：
d c (q k )＝-||(I-M c )(fθ(q k )-μ c )|| 2
wherein Mc ＝P c P c T ，μ c Embedding features S for source domain support sets s Average value of (2);
3.2.4 Computing source domain query set embedding feature f using Softmax function θ (Q s ) Sample q of (2) k Probability of belonging to class c
3.2.5 I-th base P of computation source domain subspace i And the j-th base P j Projection distance between
wherein Representation matrix P i T P j Sum of squares of all elements in (i) F Representing the F norm;
3.2.6 Calculating the source domain subspace loss function l based on the results of steps 3.2.4) and 3.2.5) S ：
wherein Ms Query set Q for source domain s Number of samples, lambda s For parameters that can be adjusted according to the experiment, λ is selected in this example s ＝0.03；
3.3 Using and calculating source domain subspace loss l S The same method calculates the embedding characteristic f of the source domain support set θ (S s ) Source domain support loss in source domain subspace
3.3.1 Computing source domain support set embedding feature f θ (S s ) Samples of (3)In the source domain subspace P c Projection distance in (a)
wherein Mc ＝P c P c T ，μ c Embedding features S for source domain support sets s Average value of (2);
3.3.2 Computing source domain support set embedding feature f using Softmax function θ (S s ) Samples of (3)Probability of belonging to class c
3.3.3 Calculating source domain support loss based on the results of steps 3.3.1) and 3.3.2)
wherein Support set S for source domain s Sample number,/->For parameters which can be adjusted experimentally, in this example, but not limited to +.>
3.4 Embedding features f using a target domain support set θ (S t ) Constructing a target domain subspace, and calculating a source domain subspace loss l by adopting and S the same method calculates the embedded feature f of the target domain query set θ (Q t ) Target domain subspace loss in target domain subspace/ T ：
3.4.1 Embedding feature f into target domain support set θ (S t ) Singular value decomposition is performed: wherein Xd Representing a target domain support set S t Class d, B d Called left singular vector, ">Called right singular vectors, T represents the transpose, Σ d Is a singular value matrix;
3.4.2 Selecting left singular vector B) d Form a truncated matrix P from the first n dimensions of (1) d ，P d A base that can be considered a target domain subspace;
3.4.3 Calculating target domain query set embedding characteristics f θ (Q t ) Sample q of (2) l In the target domain subspace P d Projection distance d of (2) d (q l )：
d d (q l )＝-||(I-M d )(f θ (q l )-μ d )|| 2
wherein μ d Embedding feature f for target domain support set θ (S t ) Average value of (2);
3.4.4 Calculating target domain query set embedding feature f using Softmax function θ (Q t ) Sample q of (2) l Probability of belonging to class d
3.4.5 Computing the u-th base P of the target domain subspace u And v th group P v Projection distance between
wherein Representation matrix->Sum of squares of all elements in (i) F Representing the F norm;
3.4.6 Calculating the target domain subspace loss function l based on the results of (3.3.4) and (3.3.5) D ：
wherein Mt Query set Q for target domain t Lambda of the number of samples of (1) t For adjustable parameters according to experimental setup, λ is selected in this example but not limited to t ＝0.03。
3.5 Using and calculating source domain subspace loss l S The same method calculates the embedded feature f of the target domain support set θ (S t ) Target domain support loss in target domain subspace
3.5.1 Computing target domain support set embedding feature f θ (S t ) Samples of (3)In the target domain subspace P d Projection distance +.>
wherein μ d Embedding feature f for target domain support set θ (S t ) Average value of (2);
3.5.2 Computing target domain support set embedding feature f using Softmax function θ (S t ) Samples of (3)Probability of belonging to class d->
3.5.3 Calculating target domain support loss based on the results of (3.5.4) and (3.5.5)
wherein Support set S for target domain t Sample number of>For the adjustable parameters according to the experimental setup, in this example, but not limited to +.>
And 4, performing contrast learning on the source domain support set data.
The contrast learning is a self-supervision learning method, and can further improve the classification precision by making a model according to the characteristics of similarity and diversity Xi Gaojie between data under the condition of no label, and the implementation steps comprise the following steps:
4.1 Construction of a contrast set for support set contrast learning, i.e. a source domain support set S containing K samples of class C, each class s Dividing into K groups, each group having C samples, wherein any two groups can be considered as a group of comparison groups; in the comparison group, each sample has only one positive sample from the same class, the other 2C-2 samples are negative samples, and the two comparison groups can be respectively expressed as:
S 1 ＝{x 1 ,x 3 ,...,x 2k -1,...,x 2C-1 }
S 2 ＝{x 2 ,x 4 ,...,x 2k ,...,x 2C }
wherein x2k-1 And x 2k Belongs to the same class;
4.2 Inputting the contrast groups into the feature extractor f respectively θ Obtaining embedded features of the comparison set and />
4.3 Calculating embedded features of a comparison set and />Noise contrast estimation loss function of medium element i,j ：
wherein s(f θ (x i ),f θ (x j ) (d) represents f θ (x i ) And f θ (x j ) Similarity measure between, τ is a temperature coefficient, selected in this example but not limited to τ=0.5;
4.4 Calculating the final contrast loss l CL ：
wherein l2k-1,2k Representation ofAnd->Noise contrast estimation loss function of medium element, l 2k,2k-1 Representation->And->The noise contrast of the medium element estimates the loss function.
And 5, constructing a domain discriminator.
Because the source domain data and the target domain data have great differences, how to solve the domain offset and realize the domain alignment are key problems to be solved urgently. The current common method is transfer learning, namely training a model in source domain data and then transferring to target domain data for classification, but the method needs fine adjustment and comprises complex setting and adjustment; the example adopts a domain discriminator based on the generation-countermeasure idea to project the fusion characteristics of the source domain and the target domain into a public space to form an integral model so as to realize domain alignment, thereby reducing training cost;
referring to fig. 3, the implementation of this step includes the following:
5.1 Five full-connection layers are selected, wherein the input characteristic quantity of the first four full-connection layers is 1024, and the output characteristic quantity is 1024; the fifth full connection layer has 1024 input features and 1 output features;
5.2 Four identical ReLU nonlinear active layers are selected, and the function of the four identical ReLU nonlinear active layers is ReLU (x) =max (x, 0);
5.3 Four identical dropout layers are selected, each layer being used to randomly set 50% of the input data to 0;
5.4 A first full connection layer, a first ReLU nonlinear activation layer and a first dropout layer are sequentially connected to form a first full connection block; sequentially connecting the second full-connection layer, the second ReLU nonlinear activation layer and the second dropout layer to form a second full-connection block; sequentially connecting a third full connection layer, a third ReLU nonlinear activation layer and a third dropout layer to form a third full connection block; sequentially connecting a fourth full connection layer, a fourth ReLU nonlinear activation layer and a fourth dropout layer to form a fourth full connection block;
5.5 A Softmax activation function selected as a function of wherein xi Representing the element in input x, Σ j exp(x i ) Representing a normalization term;
5.6 A first full connecting block, a second full connecting block, a third full connecting block, a fourth full connecting block and a fifth full connecting layer, and the Softmax activation layer is sequentially connected to form a domain discriminator f D 。
Step 6, calculating the domain discrimination loss l of the domain discriminator D 。
6.1 Loss of source domain subspace/ S Loss of source domain supportSource domain support set embedding feature f θ (S s ) And source domain query set embedding feature f θ (Q s ) Fusion is performed to obtain a source domain fusion feature (T (h s ))；
6.1.1 Using Softmax function to loss of source domain subspace/l S Conversion into Source Domain query results g s1 ：
wherein Representation l S Element of (a)>Representing a normalization term;
6.1.2 Source domain support loss using Softmax functionConversion to Source Domain support results g s2 ：
wherein Representation->Element of (a)>Representing a normalization term;
6.1.3 Supporting result g by source domain s2 And source domain query result g s1 Output result g for source domain connected in row direction s ；
6.1.4 Embedding the source domain support set into the feature f) θ (S s ) And source domain query set embedding feature f θ (Q s ) Connected in the row direction as source domain output features f s ；
6.1.5 Using multi-linear mapping to output result g to source domain s And source domain output feature f s Fusion is performed to obtain a source domain fusion feature (T (h s ))：
wherein , and />Respectively f s and gs Dimension of->Represents the pair f s and gs Carrying out outer accumulation;indicating random sampling, wherein +.is the element product +.> and />Representing two random matrices that are sampled only once and then fixed from changing during the training phase;
6.2 Loss of target domain subspace/ T Target domain support set embedding feature f θ (S t ) Target domain query set embedding feature f θ (Q t ) Fusion is performed to obtain a target domain fusion feature (T (h) t ))：
6.2.1 Using Softmax function to lose/lose target domain subspace D Conversion into target Domain query results g t1 ：
wherein Representation l S Element of (a)>Representing a normalization term;
6.2.2 Using Softmax function to lose target domain supportConversion to target Domain support results g t2 ：
wherein Representation->Element of (a)>Representing a normalization term;
6.2.3 Supporting the result g by the target domain t2 And target domain query result g t1 Output result g for target domain connected in row direction t ；
6.2.4 Embedding the target domain support set into the feature f) θ (S t ) And target domain query set embedding feature f θ (Q t ) Connecting in the row direction as target domain output feature f t ；
6.2.5 Using multi-linear mapping to output result g from target domain t And a target domain output feature f t Fusion into target Domain fusion features (T (h) t ))：
wherein , and />Respectively f t and gt Dimension of->Represents the pair f t and gt Carrying out outer accumulation;indicating random sampling, wherein +.is the product of the elements, and />Representing two random matrices, sampled only once, and then fixed for no longer changing during the training phase.
6.3 Source domain fusion feature (T (h) s ) Fusion features with target domain (T (h) t ) Common input domain discriminator f) D In the method, the domain discrimination loss l is calculated D ：
Wherein D, T respectively represent the fusion characteristics of the domain discriminator and the characteristic extractor, and x s ，x t Representing a source domain sample and a target domain sample, respectively, P s (x)，P t (x) Representing the source domain distribution and the target domain distribution, respectively, D (T (h s ) Representing a discriminator f D Prediction source domain fusion features (T (h) s ) A) the probability of originating from the source domain,representation discriminator f D Predicting target domain fusion featuresThe probability derived from the target domain, E, is the desired output.
And 7, calculating a total loss function of the source domain and the target domain, and training the feature extractor.
7.1 Calculating the subspace loss l from the source domain S Target domain subspace loss l T Loss of contrast l CL Domain discrimination loss l D Total loss function of composition: l=λ 1 l S +λ 2 l T +λ 3 l CL +l D
wherein λ1 、λ 2 、λ 3 Respectively representing source domain subspace loss l S Target domain subspace loss l T Loss of contrast l CL Domain discrimination loss l D The values of which are set according to experiments, in this example are selected from but not limited to lambda 1 :λ 2 :λ 3 ＝2:0.1:1；
7.2 Calculating gradients of residual block and convolutional layer parameters in the feature extractor; calculating the parameter gradient of the full connection layer in the domain discriminator;
7.3 Updating parameters of the feature extractor and domain discriminator using an optimization algorithm, in this example selected but not limited to adam optimization, based on the total loss/and the calculated gradients;
7.4 By iteratively performing the calculation of the total loss, the calculation of the gradient and the updating of the parameters, the feature extractor and the domain discriminator are continuously adjusted to obtain an updated feature extractor f θ ' and updated domain authentication f D '；
7.5 Using the updated feature extractor f once every E training algebra θ ' categorize the test set data, in this example selected but not limited to e=500;
7.6 Repeating steps 3, 4, 6 and 7.1) to 7.4) until the total loss l tends to stabilize or no longer significantly decreases, resulting in a trained feature extractor f θ '。
And 8, classifying by using the test set data.
8.1 Training set T) r Input to the mapping layer and trained feature extractor f θ In' get training set embedding feature f θ '(T r ) By utilizing the characteristic f θ '(T r ) Building training set subspaces；
8.1.1 Embedding features f) into training set θ '(T r ) Singular value decomposition is performed: f (f) θ '(X e )＝B e Σ e V e T, wherein Xe Representing training set T r Class e, B of (2) e Called left singular vectors, V e T Called right singular vectors, T represents the transpose, Σ e Is a singular value matrix;
8.1.2 Selecting left singular vector B) e Form a truncated matrix P from the first n dimensions of (1) e And P is taken e A base treated as a training set subspace; selected in this example but not limited to n=2;
8.2 To test set T) e Input to the mapping layer and trained feature extractor f θ In' get test set embedded feature f θ '(T e ) And calculates its test subspace loss in the training set subspace, l':
8.2.1 Calculating test set embedded features f θ '(T e ) Sample q of (3) m In training set subspace P e Projection distance d of (2) e (q m )：
d e (q m )＝-||(I-M e )(f θ '(q m )-μ e )|| 2
wherein μ e Embedding feature f for training set θ '(T r ) Average value of (2);
8.2.2 Calculating test set embedding characteristics f by Softmax function θ '(T e ) Sample q of (3) m Probability belonging to class m
8.2.3 A) computing the training set subspaceRadical P a And the b-th group P b Projection distance between
wherein Representation matrix->Sum of squares of all elements in (i) F Representing the F norm;
8.2.4 Calculating a test set subspace loss l' from the results of steps 8.2.2) and 8.2.3):
wherein M' is the test set T r The number of samples, λ', is a parameter that is adjustable according to the experiment;
8.3 Taking the smallest category in the test subspace loss l' as the prediction category to finish the classification task.
The sequence numbers of the steps are for clearly illustrating the implementation scheme of the invention, and the sequence numbers are not limited.
The technical effects of the present invention will be further described with reference to simulation experiments.
1. Simulation conditions and content:
the running environment of the simulation experiment is: windows 10 professional 64-bit operating system, CPU is Intel (R) Core (TM) i5-12490F 3.00GHz, memory is 16GB, graphics card is NVIDIA GeForce RTX 3060, and the operating environment is Cuda11.8, python 3.8 and pytorch 2.0.0.
The hyperspectral image dataset used in the simulation experiment comprises a source domain dataset and a target domain dataset, wherein the source domain dataset and the target domain dataset are the same as the source domain dataset;
the source domain data set adopts a Chikusei data set acquired by a visible light/near infrared camera Hyperspec-VNIR-C in Chikusei in Japan;
the target domain data sets were the Indian pins data set collected on the Indian pins test site in northwest of Indiana and the Salinas data set collected over Salinas Valley in California, USA, respectively, and the Pavia unitary data set obtained by photographing at university of Parvia in North Italy using ROSIS hyperspectral remote sensing satellites.
The Chikusei dataset image size was 2517 x 2335 with 128 spectral bands, containing 19 classes of features, the class and number of each class of features as shown in Table 1.
The Indian pins dataset has an image size of 145 x 145 and 200 spectral bands, containing 16 types of features, and the type and number of each type of feature are shown in table 2.
The Pavia universality dataset image size is 610 x 340, with 103 spectral bands, containing 9 types of features, and the type and number of each type of feature are shown in table 3.
The Salinas dataset image size was 512 x 217 with 204 spectral bands, containing 16 classes of features, class and number of features per class, as shown in table 4.
TABLE 1 Chikusei sample categories and amounts
/>
TABLE 2 Indian pins sample class and quantity
| Class label | Ground object category | Quantity of |
| 1 | Alfalfa | 46 |
| 2 | Corn-notill | 1428 |
| 3 | Corn-mintill | 830 |
| 4 | Corn | 237 |
| 5 | Grass-pasture | 483 |
| 6 | Grass-trees | 730 |
| 7 | Grass-pasture-mowed | 28 |
| 8 | Hay-windrowed | 478 |
| 9 | Oats | 20 |
| 10 | Soybean-nottill | 972 |
| 11 | Soybean-mintill | 2455 |
| 12 | Soybean-clean | 593 |
| 13 | Wheat | 205 |
| 14 | Woods | 1265 |
| 15 | Buildings-grass-trees-drives | 386 |
| 16 | Stone-steel-towers | 93 |
Table 3 Pavia university sample class and quantity
| Class label | Ground object category | Quantity of |
| 1 | Asphalt | 6631 |
| 2 | Meadows | 18649 |
| 3 | Gravel | 2099 |
| 4 | Trees | 3064 |
| 5 | Sheets | 1345 |
| 6 | Baresoil | 5029 |
| 7 | Bitumen | 1330 |
| 8 | Bricks | 3682 |
| 9 | Shadows | 947 |
TABLE 4 Salinas sample class and quantity
| Class label | Ground object category | Quantity of |
| 1 | Brocoli_green_weeds_1 | 2009 |
| 2 | Brocoli_green_weeds_2 | 3726 |
| 3 | Fallow | 1976 |
| 4 | Fallow_rough_plow | 1394 |
| 5 | Fallow_smooth | 2678 |
| 6 | Stubble | 3959 |
| 7 | Celery | 3579 |
| 8 | Grapes_untrained | 11271 |
| 9 | Soil_vinyard_develop | 6203 |
| 10 | Corn_senesced_green_weeds | 3278 |
| 11 | Lettuce_romaine_4wk | 1068 |
| 12 | Lettuce_romaine_5wk | 1927 |
| 13 | Lettuce_romaine_6wk | 916 |
| 14 | Lettuce_romaine_7wk | 1070 |
| 15 | Vinyard_untrained | 7268 |
| 16 | Vinyard_vertical_trellis | 1807 |
2. Simulation content and result analysis:
simulation 1. Classification was performed on hyperspectral images described in Table 2 using the present invention and three methods RN-FSC, DCFSL, gia-CFSL, respectively, and the respective overall classification accuracy OA, average classification accuracy AA and kappa coefficients were compared, and the results are shown in Table 5.
TABLE 5 comparison of the results of three prior art with the present invention on Indian pins
/>
Simulation 2. Classification was performed on hyperspectral images described in Table 3 using the present invention and the three methods of the prior art, RN-FSC, DCFSL, gia-CFSL, respectively, and the respective overall classification accuracy OA, average classification accuracy AA, and kappa coefficients were compared, and the results are shown in Table 6.
Table 6 results of prior art comparison with the present invention on Pavia universality
Simulation 3. Classification was performed on hyperspectral images described in Table 4 using the present invention and the three methods RN-FSC, DCFSL, gia-CFSL, respectively, and the respective overall classification accuracy OA, average classification accuracy AA and kappa coefficients were compared, and the results are shown in Table 7.
Table 7 results of three prior art comparisons with the present invention on Salinas
Simulation 4, the time required for the respective runs was compared with the classification of the hyperspectral images described in tables 2, 3 and 4 by the present invention and the three methods of the prior art RN-FSC, DCFSL, gia-CFSL, respectively, and the results are shown in table 8.
TABLE 8 comparison of the results of the invention with three prior art methods over the time(s) required for operation
The three prior art sources in tables 5, 6, 7, 8 are as follows:
the RN-FSC refers to a relational network-based method RN-FSC proposed by Gao et al in Deep relation network for hyperspectral image few-shot classification, remote sens.vol.12, no.6, p.923, mar.2020 ".
The DCFSL refers to a method DCFSL based on condition-based domain adaptation proposed by Li et al in Deep cross-domain few-shot learning for hyperspectral image classification, IEEE Trans. Geosci. Remote sens, vol.60, pp.1-18,2021.
The Gia-CFSL refers to a graph information aggregation-based method Gia-CFSL as proposed by Zhang et al in Graph information aggregation cross-domain few-shot learning for hyperspectral image classification, "IEEE Trans.Neal NetworkLearn. Syst., early access, jun.30, 2022".
It can be seen from tables 5, 6 and 7 that the overall classification accuracy and the average classification accuracy of the method of the present invention are higher than those of the existing method on the three data sets, and the consistency of the method is higher than that of the existing method except for Salinas.
As can be seen from table 8, the time required for classification of the present invention over three data sets is relatively low compared to the prior art.
The simulation result shows that the invention not only improves the distinguishing capability and generalization performance of the classifier, but also improves the utilization rate of embedded feature information through the subspace classification adopted. Meanwhile, the invention realizes domain alignment by using the domain discriminator to project the fusion characteristics of the source domain and the target domain to the public space, thereby reducing training cost. In addition, the method and the device train the source domain support set data by using contrast learning, so that the classification precision of the cross-domain small sample is further improved.
Claims (10)
1. A small sample cross-domain hyperspectral image classification method based on contrast learning and subspace is characterized by comprising the following steps:
(1) Acquiring a hyperspectral data set and dividing:
(1a) G hyperspectral data sets are obtained from a public website, one data set is used as a source domain, and the remaining G-1 data sets are used as target domains;
(1b) C categories are selected from the source domain data set, and K source domain samples are selected as a source domain support set S from each category s Then N other source domain samples are selected to form a source domain query set Q s ；
(1c) H tagged categories are selected from the target domain data set, and K target domain tagged samples are selected from each category as a target domain support set S t N other target domains are selected again by each class and have label samples to form a target domain query set Q t Support set S with target domain t Query set Q with target domain t Together form a training set T r The method comprises the steps of carrying out a first treatment on the surface of the Taking the rest target domain unlabeled sample as a test set T e ；
(2) Constructing a mapping layer comprising a two-dimensional convolution layer and a BatchNorm2d layer; constructing a feature extractor f comprising two residual blocks, two max-pooling layers, one convolutional layer and one flame layer θ ；
(3) Subspace classification is carried out on embedded features of a source domain and a target domain:
(3a) Support set S of source domain s Source domain query set Q s Target domain support set S t Target domain query set Q t Respectively input to the mapping layer and the feature extractor f θ Obtaining the embedding characteristic f of the source domain support set θ (S s ) Embedding feature f in source domain query set θ (Q s ) Target domain support set embedding feature f θ (S t ) Target domain query set embedding feature f θ (Q t )；
(3b) Embedding feature f with source domain support set θ (S s ) Constructing a source domain subspace and calculating a source domain query set embedding feature f θ (Q s ) Source domain subspace loss in source domain subspace/ S ；
(3c) Computing source domain support set embedding features f θ (S s ) Source domain support loss in source domain subspace
(3d) Embedding feature f with target domain support set θ (S t ) Constructing a target domain subspace and calculating an embedded feature f of a target domain query set θ (Q t ) Target domain subspace loss in target domain subspace/ T ；
(3e) Computing target domain support set embedding features f θ (S t ) Target domain support loss in target domain subspace
(4) Support set S of source domain s Dividing into C contrast groups, and calculating contrast loss l of the input mapping layer and the feature extractor CL ；
(5) Constructing a domain discriminator f comprising five full connection layers, four ReLU nonlinear activation functions, four dropout layers, a Softmax activation function D ；
(6) Calculating a domain discrimination penalty l D ：
(6a) Loss of source domain subspace/ S Loss of source domain supportSource domain support set embedding feature f θ (S s ) Embedding feature f in source domain query set θ (Q s ) Fusion is performed to obtain a source domain fusion feature (T (h s ))；
(6b) Loss of target domain subspace/ T Loss of target domain supportTarget domain support set embedding feature f θ (S t ) Target domain query set embedding feature f θ (Q t ) Fusion is performed to obtain a target domain fusion feature (T (h) t ))；
(6c) The fusion characteristics of the source domain and the target domain (T (h s ))、(T(h t ) Input field discriminator f) D Medium calculation domain discrimination loss l D ：
Wherein D, T respectively represent the fusion characteristics of the domain discriminator and the characteristic extractor, and x s ，x t Representing a source domain sample and a target domain sample, respectively, P s (x)，P t (x) Representing the source domain distribution and the target domain distribution, respectively, D (T (h s ) Representing a discriminator f D Prediction source domain fusion features (T (h) s ) A) the probability of originating from the source domain,representation discriminator f D Prediction target Domain fusion feature->The probability from the target domain, E, is the expected output;
(7) Computing a total loss function to train the feature extractor:
calculating the subspace loss l of the source domain S Target domain subspace loss l T Loss of contrast l CL Domain discrimination loss l D Total loss function of composition: l=λ 1 l S +λ 2 l T +λ 3 l CL +l D, wherein λ1 、λ 2 、λ 3 Respectively is l S 、l T 、l CL Is provided;
feature extractor f by back propagation θ Iterative training is carried out until the total loss function converges, and a trained feature extractor f is obtained θ '；
(8) Classifying by using test set data:
(8a) Training set T r Input to the mapping layer and trained feature extractor f θ In' get training set embedding feature f θ '(T r ) By utilizing the characteristic f θ '(T r ) Constructing a training set subspace;
(8b) Test set T e Input to the mapping layer and trained feature extractor f θ In' get test set embedded feature f θ '(T e ) And calculating a test subspace loss l' thereof in the training set subspace;
(8c) And taking the smallest category in the test subspace loss l' as a prediction category to finish the classification task.
2. The method according to claim 1, characterized in that: in the step (2), a two-dimensional convolution layer and a BatchNorm2d layer in the mapping layer are formed, and the structural parameters are as follows:
the two-dimensional convolution layer has the input channel number of the dimension of input data, the output channel number of 100, the convolution kernel size of 1 multiplied by 1 and the step length of 1;
the BatchNorm2d layer is used for two-dimensional batch normalization, and the formula isWhere x represents the input data, mean (x) represents the mean, var (x) represents the variance, eps is a minimum, and γ, β are two different values of the adjustment parameters, respectively.
3. The method according to claim 1, characterized in that: the feature extractor f is constituted in step (2) θ Two residual blocks, two max-pooling layers, one convolution layer, its structureThe parameters are as follows:
the feature extractor f θ The device is formed by sequentially connecting a first residual block, a first pooling layer, a second residual block, a second pooling layer and a convolution layer;
the two residual blocks have the same structure, and each residual block comprises 3 convolution blocks and 3 ReLU layers: the first convolution block, the first ReLU layer, the second convolution block, the second ReLU layer, the third convolution block and the third ReLU layer are sequentially connected, and the output of the first ReLU layer is connected with the third convolution block in a residual way; each convolution block comprises a 3D convolution layer and a battnorm 3D layer; the convolution kernel of the 3D convolution layer has the size of 3 multiplied by 3, the step length is 1, and the filling is 1; the function of each ReLU layer is ReLU (x) =max (x, 0);
The two largest pooling layers have different parameters, namely the pooling core size of the first pooling layer is 4 multiplied by 2, the filling is 0 multiplied by 1, and the step length is 4 multiplied by 2; the size of the pooling core of the second pooling layer is 4 multiplied by 2, the filling is 4 multiplied by 2, and the step size is 2 multiplied by 1;
the convolution layer has an output channel number of 32, convolution kernel size is 3× 3X 3.
4. The method according to claim 1, characterized in that: embedding feature f in step (3 b) using source domain support set θ (S s ) Constructing a source domain subspace and calculating a source domain query set embedding feature f θ (Q s ) Source domain subspace loss in source domain subspace/ S The method comprises the following steps:
(3b1) Embedding feature f into source domain support set θ (S s ) Singular value decomposition wherein Xc Representing a source domain support set S s Class c, B c Called left singular vector, ">Called right singular vectors, T represents the transpose, Σ c Is a singular value matrix;
(3b2) Selecting left singular vectors B c Form a truncated matrix P from the first n dimensions of (1) c And P is taken c A base regarded as a source domain subspace;
(3b3) Computing source domain query set embedding features f θ (Q s ) Sample q of (3) k In the source domain subspace P c Projection distance d of (2) c (q k )：
d c (q k )＝-||(I-M c )(f θ (q k )-μ c )|| 2
wherein μ c Embedding features S for source domain support sets s Average value of (2);
(3b4) Computing source domain query set embedding feature f using Softmax function θ (Q s ) Sample q of (2) k Probability of belonging to class c
(3b5) Computing the ith base P of the source domain subspace i And the j-th base P j Projection distance between
wherein Representation matrix P i T P j Sum of squares of all elements in (i) F Representing the F norm;
(3b6) Calculating a source domain subspace loss function l from the results of (3 b 4) and (3 b 5) S ：
wherein Ms Query set Q for source domain s Number of samples, lambda s Is a parameter that can be adjusted according to the experiment.
5. The method according to claim 1, characterized in that: in step (4), the source domain support set S s Dividing into C contrast groups, and calculating contrast loss l of the input mapping layers and the feature extractor of the contrast groups CL The method comprises the following steps:
(4a) Source domain support set S containing class C and K samples of each class s Dividing into K groups, each group having C samples, wherein any two groups can be regarded as a comparison group, in which each sample has only one positive sample from the same class, the remaining 2C-2 samples are negative samples, and the two comparison groups can be respectively expressed as S 1 and S2 ：
S 1 ＝{x 1 ,x 3 ,...,x 2k-1 ,...,x 2C-1 }
S 2 ＝{x 2 ,x 4 ,...,x 2k ,...,x 2C }
wherein x2k-1 And x 2k Belongs to the same class;
(4b) The contrast groups are respectively input into a feature extractor f θ Obtaining embedded features of the comparison set and />
(4c) Computing embedded features for a comparison set and />Noise contrast estimation loss function of medium element i,j ：
wherein s(f θ (x i ),f θ (x j ) (d) represents f θ (x i ) And f θ (x j ) Similarity measurement between the two, τ is a temperature coefficient;
(4d) Calculate the final contrast loss l CL ：
wherein l2k-1,2k Representation ofAnd->Noise contrast estimation loss function of medium element, l 2k,2k-1 Representation->And->The noise contrast of the medium element estimates the loss function.
6. The method according to claim 1, characterized in that: step (5) constituting a domain discriminator f D Four sulu nonlinear active layers, four dropout layers, one Softmax active layer, the structural parameters of which include the following:
the domain discriminator f D The device comprises a first full connecting block, a second full connecting block, a third full connecting block, a fourth full connecting block and a fifth full connecting layer, wherein the Softmax active layer is sequentially connected, and the four full connecting blocks are sequentially connected by the full connecting layer, the ReLU nonlinear active layer and the dropout layer;
the five full-connection layers have 1024 input characteristic numbers and 1024 output characteristic numbers; the fifth full connection layer has 1024 input features and 1 output features;
the four ReLU nonlinear active layers are identical, and the function of the four ReLU nonlinear active layers is ReLU (x) =max (x, 0);
the four dropout layers are identical and represent that part of the input features are randomly set to 0 and the parameters are set to 0.5;
The Softmax activation function is as follows wherein xi Representing the element in input x, Σ j exp(x i ) Representing the normalized term.
7. The method according to claim 1, characterized in that: step (6 a) of losing the subspace of the source domain S Source domain support set embedding feature f θ (S s ) Source domain query set embedding feature f θ (Q s ) Fusing to obtain source domain fusion characteristicsSign (T (h) s ) The method comprises the following steps:
(6a1) Source domain subspace loss/using a Softmax function S Conversion into Source Domain query results g s1 ：
wherein Representation l S Element of (a)>Representing a normalization term;
(6a2) Embedding source domain support sets into feature f θ (S s ) Projection to source domain subspace to calculate loss, and calculation to obtain source domain support result g by using Softmax function s2 ；
(6a3) Supporting the result g by the source domain s2 Query result g with source domain s1 Output result g for source domain connected in row direction s ；
(6a4) Embedding source domain support sets into feature f θ (S s ) Source domain query set embedding feature f θ (Q s ) Connected in the row direction as source domain output features f s ；
(6a5) Outputting result g to source domain by adopting multi-linear mapping s And source domain output feature f s Fusion is performed to obtain a source domain fusion feature (T (h) s ))：
wherein , and />Respectively f s and gs Dimension of->Represents the pair f s and gs Carrying out outer accumulation;indicating random sampling, wherein +.is the product of the elements, and />Representing two random matrices that are sampled only once and then fixed from changing during the training phase.
8. The method according to claim 1, characterized in that: loss of target domain subspace/in step (6 b) T Target domain support set embedding feature f θ (S t ) Target domain query set embedding feature f θ (Q t ) Fusion is performed to obtain a target domain fusion feature (T (h) t ) The steps are as follows:
(6b1) Target domain subspace loss/using a Softmax function D Conversion into target Domain query results g t1 ：
wherein Representation l S Element of (a)>Representing a normalization term;
(6b2) Embedding a target domain support set into feature f θ (S t ) Projecting the target domain support result g to a target domain subspace to calculate loss, and calculating the target domain support result g by using a Softmax function t2 ；
(6b3) Supporting the result g by the target domain t2 Query result g with target domain t1 Output result g for target domain connected in row direction t ；
(6b4) Embedding a target domain support set into feature f θ (S t ) Target domain query set embedding feature f θ (Q t ) Connecting in the row direction as target domain output feature f t ；
(6b5) Outputting the result g from the target domain by adopting multi-linear mapping t And a target domain output feature f t Fusion into target Domain fusion features (T (h) t ))：
wherein , and />Respectively f t and gt Dimension of->Represents the pair f t and gt Carrying out outer accumulation; Indicating random sampling, wherein +.is the product of the elements, and />Representing two random matrices, sampled only once, and then fixed for no longer changing during the training phase.
9. The method according to claim 1, characterized in that: in step (8 a) training set feature f is used θ '(T r ) The training set subspace is constructed, and the steps comprise the following steps:
(8a1) Embedding feature f into training set θ '(T r ) Singular value decomposition is performed: f (f) θ '(X e )＝B e Σ e V e T, wherein Xe Representing training set T r Class e, B of (2) e Called left singular vectors, V e T Called right singular vectors, T represents the transpose, Σ e Is a singular value matrix;
(8a2) Selecting left singular vectors B e Form a truncated matrix P from the first n dimensions of (1) e And P is taken e Treated as the basis for a training set subspace.
10. The method according to claim 1, characterized in that: calculating test set embedding characteristics f in step (8 b) θ '(T e ) A test set subspace loss l' in the training set subspace, comprising the steps of:
(8b1) Computing test set embedding characteristics f θ '(T e ) Sample q of (3) m In training set subspace P e Projection distance d of (2) e (q m )：
d e (q m )＝-||(I-M e )(f θ '(q m )-μ e )|| 2
wherein μ e Embedding feature f for training set θ '(T r ) Average value of (2);
(8b2) Calculation of test set embedding characteristics f by Softmax function θ '(T e ) Sample q of (3) m Probability belonging to class m
(8b3) Computing the a-th base P of the training set subspace a And the b-th group P b Projection distance between
wherein Representation matrix->Sum of squares of all elements in (i) F Representing the F norm;
(8b4) Calculating a test set subspace loss l' from the results of (8 b 2) and (8 b 3):
wherein M' is the test set T r The number of samples, λ', is an experimentally adjustable parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310979029.8A CN116977744A (en) | 2023-08-04 | 2023-08-04 | Small sample cross-domain hyperspectral image classification method based on contrast learning and subspace |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310979029.8A CN116977744A (en) | 2023-08-04 | 2023-08-04 | Small sample cross-domain hyperspectral image classification method based on contrast learning and subspace |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116977744A true CN116977744A (en) | 2023-10-31 |
Family
ID=88483003
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310979029.8A Pending CN116977744A (en) | 2023-08-04 | 2023-08-04 | Small sample cross-domain hyperspectral image classification method based on contrast learning and subspace |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116977744A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118097315A (en) * | 2024-04-26 | 2024-05-28 | 中国科学技术大学 | Small sample image classification method based on contrast learning |
-
2023
- 2023-08-04 CN CN202310979029.8A patent/CN116977744A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118097315A (en) * | 2024-04-26 | 2024-05-28 | 中国科学技术大学 | Small sample image classification method based on contrast learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108388927B (en) | Small sample polarization SAR terrain classification method based on deep convolution twin network | |
| WO2022160771A1 (en) | Method for classifying hyperspectral images on basis of adaptive multi-scale feature extraction model | |
| CN105138973B (en) | The method and apparatus of face authentication | |
| CN108846426B (en) | Polarization SAR classification method based on deep bidirectional LSTM twin network | |
| CN108280396B (en) | Hyperspectral image classification method based on depth multi-feature active migration network | |
| CN105608471A (en) | Robust transductive label estimation and data classification method and system | |
| CN111783865B (en) | Hyperspectral classification method based on space spectrum neighborhood embedding and optimal similarity graph | |
| CN111832414A (en) | Animal counting method based on graph regular optical flow attention network | |
| CN116977744A (en) | Small sample cross-domain hyperspectral image classification method based on contrast learning and subspace | |
| Lin et al. | Deep transfer learning for hyperspectral image classification | |
| Tanwar et al. | Deep learning-based hybrid model for severity prediction of leaf smut sugarcane infection | |
| CN115311502A (en) | Remote sensing image small sample scene classification method based on multi-scale double-flow architecture | |
| CN114937173A (en) | Hyperspectral image rapid classification method based on dynamic graph convolution network | |
| AU2015218184B2 (en) | Processing hyperspectral or multispectral image data | |
| Sehree et al. | Olive trees cases classification based on deep convolutional neural network from unmanned aerial vehicle imagery | |
| Sjahputera et al. | Clustering of detected changes in high-resolution satellite imagery using a stabilized competitive agglomeration algorithm | |
| Song et al. | Multi-source remote sensing image classification based on two-channel densely connected convolutional networks. | |
| CN113627240B (en) | Unmanned aerial vehicle tree species identification method based on improved SSD learning model | |
| Mei et al. | Cascade residual capsule network for hyperspectral image classification | |
| Farooque et al. | Red-green-blue to normalized difference vegetation index translation: a robust and inexpensive approach for vegetation monitoring using machine vision and generative adversarial networks | |
| Sreedevi et al. | Development of weighted ensemble transfer learning for tomato leaf disease classification solving low resolution problems | |
| Alzhanov et al. | Crop classification using UAV multispectral images with gray-level co-occurrence matrix features | |
| CN117079017A (en) | Credible small sample image identification and classification method | |
| Huang et al. | Hyperspectral Image Classification via Cross-Domain Few-Shot Learning With Kernel Triplet Loss | |
| CN113836996B (en) | Hyperspectral remote sensing image part migration method and hyperspectral remote sensing image part migration system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |