CN112651317B - Hyperspectral image classification method and system for sample relation learning - Google Patents
Hyperspectral image classification method and system for sample relation learning Download PDFInfo
- Publication number
- CN112651317B CN112651317B CN202011509345.1A CN202011509345A CN112651317B CN 112651317 B CN112651317 B CN 112651317B CN 202011509345 A CN202011509345 A CN 202011509345A CN 112651317 B CN112651317 B CN 112651317B
- Authority
- CN
- China
- Prior art keywords
- sample
- training
- samples
- model
- classified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
Abstract
A sample relation learning hyperspectral image classification method and a system are disclosed, wherein the method comprises the steps of extracting single-sample features by adopting a deep neural network, and extracting common features of multiple samples by utilizing the common features; then, by adopting feature linking, the features of the samples to be classified and the common features of the single classes are linked to obtain measurement features; finally, calculating the relation score of the metric characteristics by using metric learning; calculating a loss value in a training phase based on the calculated relation evaluation set based on the common characteristics of the samples to be classified and each class, updating parameters of a model by using a back propagation algorithm, judging class information of the samples to be classified through a maximum probability value during application, and generating a classification result graph pixel by pixel; the invention can fully excavate the difference between multi-class samples, and can realize high-precision identification of different classes of information in the hyperspectral image only by a small number of labeled samples.
Description
Technical Field
The invention relates to the field of spectral image classification and identification, in particular to a method and a system for classifying hyperspectral images through sample relation learning.
Background
The hyperspectral image has the advantage of high spectral resolution, and the contained information can reflect the external quality characteristics (including size, shape and defect) and the difference of internal physical structure and chemical composition of the ground object type, and is always very important application data. Due to the abundant spectral characteristics, the method has important application values in the aspects of material classification, ground surface coverage classification, accurate agriculture, environmental monitoring and the like. The hyperspectral image has high spectrum dimensionality, and the spectra of different ground objects show the phenomena of large intra-class difference and small inter-class difference. Therefore, in order to extract key information from the hyperspectral image, a large number of labeled samples are usually required, and a classification model with high complexity is constructed. For example, in recent years, a deep neural network model has been widely applied to the field of hyperspectral image classification, and has achieved remarkable achievement.
However, deep neural network models are inherently dependent on large data drives, which makes the problem of small samples in hyperspectral images increasingly prominent. In particular, the collection of samples of hyperspectral images, which cannot be obtained from the images by visual interpretation, usually relies on the investigation of professionals in the field, which means that their sample collection work requires a lot of manpower and material resources. Therefore, under the condition of a small sample, the research of the high-precision hyperspectral image classification model has important application value.
For how to fully train the deep neural network in the case of small samples, the most direct scheme is to add samples (pseudo-labeled samples or virtual samples) by using mathematical transformation, for example, generating pseudo-labeled samples by using a self-training method, or obtaining identically distributed virtual samples by generating a countering neural network. However, the increased sample information is still limited to the information range of the small sample data space, so the performance improvement of the deep neural network is still limited, and the erroneous sample is easily generated, thereby causing an accumulative error. If the model can learn the capability of comparing sample differences from the perspective of a plurality of sample relationships, a large amount of training data can be obtained through multi-sample recombination, and the model can fully learn the multi-sample relationships in the training process: the difference between the samples to be classified and the samples in each category can play an important role when the relation is applied to a classification model.
Therefore, how to construct a deep neural network model for sample relationship learning becomes a technical problem to be solved urgently in the prior art for hyperspectral image classification and hyperspectral data application.
Disclosure of Invention
The invention aims to overcome the problem of lack of training samples in the field of hyperspectral image classification, and provides a hyperspectral image classification method and a hyperspectral image classification system for sample relation learning. The invention can fully excavate the difference between multi-class samples, and can realize high-precision identification of different classes of information in the hyperspectral image only by a small number of labeled samples.
In order to achieve the purpose, the invention adopts the following technical scheme:
a sample relation learning hyperspectral image classification method comprises
The method comprises two stages: a training method and an application method thereof,
the training method comprises the following specific steps:
target category sample set construction step S110: selecting a hyperspectral image with high quality and subjected to radiation correction from a hyperspectral image library of a certain sensor, and constructing a sample set X containing different target categories, wherein the number of each sample can be as small as 100 pixels at least;
a model supporting sample set constructing step S120, namely dividing the samples in the target category sample set X into different subsets according to categories, namely putting the samples with the same category together to form a single subset, and then using the divided subsets as elements to form a new set to form a supporting sample set S;
a step S130 of constructing a model query sample set, in which the target category sample set X is copied and expanded in equal proportion, the quantity of each category sample after expansion is ensured to be the same, and a query sample set Q is obtained train For training purposes;
a step S140 of constructing a model input example set, in which in the support sample set S, n samples S are randomly selected for each category j ＝{x j1 ,…,x jn J e {1, \8230;, C }, the set of query samples Q used in training train In (1), randomly selecting a sample x q Composing a training input instance
Repeat the aboveAnd (5) an input instance construction process is carried out, and an input instance set of model training is obtained.
Single sample feature extraction step S150: randomly selecting a number of instances from a training input instance set, for each instance
The sample in (1) comprises a support sample and a training query sample, and a deep neural network is applied
Respectively extracting the characteristics of each sample in the example to obtain the single-sample characteristics of the supporting sample:
j ∈ {1, \8230;, C }, and the single-sample features of the training query sample
Wherein
Extracting parameters for the single sample features;
a common feature extraction step S160: will now be described
Support sample characteristics for each category in
Utilizing deep neural networks
Generating common characteristics of the category
Wherein
Extracting parameters for the common features;
metrology featuresThe linking step S170: will be described in detail with reference to the following examples
The training query sample feature in (1), that is, the single sample feature of the sample to be classified
Features common to said classes
Obtaining characteristics to be measured by using link method
Wherein η is a characteristic linking parameter;
metric feature scoring step S180: the characteristics to be measured are measured
Input to a deep neural network g ψ Obtaining a relationship score representing a training query sample x q Probability values belonging to class j, where ψ is a metric parameter;
model parameter updating step S190: calculating the loss sum of the selected multiple instances by using a loss function based on the relation score, updating the parameters of the model by using a back propagation mechanism, and finally finishing the training of the model, namely continuously iterating the steps S150-S180 according to the multiple instance sets, and optimizing the characteristic extraction parameters of the single sample
Common feature extraction parameters
The characteristic link parameter η and the metric parameter ψ until the loss function of an arbitrary batch instance set steadily approaches 0.
The application method comprises the following steps:
the method comprises the steps of sequentially deducing a category label for a given hyperspectral image to be classified according to a pixel-by-pixel method, constructing a sample example to be classified according to the pixel to be classified, sequentially applying trained related steps, including a single sample feature extraction step, a common feature extraction step, a measurement feature linking step and a measurement feature scoring step, obtaining the relationship scoring set of the sample example to be classified, then obtaining category information of the pixel to be classified according to maximum scoring, and finally generating a classification result image according to the pixel-by-pixel deduction result.
The invention further discloses a sample relation learning hyperspectral image classification system which comprises
The device comprises two parts: a model training part and a model application part,
the model training part comprises the following concrete steps:
the target class sample set construction module 210: selecting a hyperspectral image with high quality and subjected to radiation correction from a hyperspectral image library of a certain sensor, and constructing a sample set X containing different target categories, wherein the number of each sample can be as small as 100 pixels at least;
the model support sample set construction module 220 divides the samples in the target category sample set X into different subsets according to categories, namely, the samples with the same category are put together to form a single subset, and then the divided subsets are used as elements to form a new set to form a support sample set S;
the model query sample set construction module 230 is to copy and expand the target category sample set X in equal proportion, so as to ensure the same number of each category sample after expansion, and obtain a query sample set Q train For training;
model input instance set construction module 240 for randomly selecting n samples S for each category in the support sample set S j ＝{x j1 ,…,x jn J ∈ {1, \ 8230;, C }, the set of query samples Q used in training train In (1), randomly selecting a sample x q Composing a training input instance
And repeating the input instance construction process to obtain an input instance set for model training.
Single sample feature extraction module 250: randomly selecting a number of instances from a training input instance set, for each instance
The sample in (1) comprises a support sample and a training query sample, and a deep neural network is applied
And respectively extracting the characteristics of each sample in the example to obtain the single sample characteristics of the supporting sample:
j ∈ {1, \8230;, C }, and the single-sample features of the training query sample
Wherein
Extracting parameters for the single sample features;
the common feature extraction module 260: will now be described
Support sample characteristics of each category in
Utilizing deep neural networks
Generating a commonality feature for the category
Wherein
Extracting parameters for the common features;
metric feature linking module 270: will now be described
The training query sample in (1), that is, the single-sample feature of the sample to be classified
Features common to said classes
Obtaining characteristics to be measured by using link method
Wherein η is a characteristic linking parameter;
metric feature scoring module 280: the characteristics to be measured are measured
Input to a deep neural network g ψ Obtaining a relationship score representing a training query sample x q Probability values belonging to class j, where ψ is a metric parameter;
model parameter update module 290: calculating the loss sum of the selected multiple instances by using a loss function based on the relation score, updating the parameters of the model by using a back propagation mechanism, and finally finishing the training of the model, namely continuously iterating the modules 250-280 according to the multiple instance sets to optimize the single-sample feature extraction parameters
Common feature extraction parameters
The feature linking parameter η and the metric parameter ψ are obtained until the loss function of any batch of the example set is stably approximated to 0, and a trained single-sample feature extraction module 250, a common feature extraction module 260 and a metric feature scoring module 280 are obtained.
The model application part comprises
The classification result image generation module 295: the method comprises the steps of classifying a given hyperspectral image to be classified in sequence according to a pixel-by-pixel method, constructing a sample example to be classified according to the pixel to be classified, obtaining the relation evaluation set of the sample example to be classified by sequentially applying trained modules, wherein the trained modules comprise a single sample feature extraction module 250, a common feature extraction module 260 and a measurement feature evaluation module 280, obtaining the category information of the pixel to be classified according to the maximum evaluation, and finally generating a classification result image according to the pixel-by-pixel inference result.
In conclusion, the invention has the following advantages:
1. the depth model of the invention only needs a small amount of marking samples, each type can be less than 100 marking pixels, and high precision is achieved;
2. the invention can obtain a large amount of training data according to a small amount of samples, and the number of the marked samples is | | | X | | |, the marked samples comprise C categories, and the number of the samples contained in each category is the same as
The size of the constructed set of input instances (i.e., the model training data set) is then
3. The method can learn the relation score between the samples to be classified and each class sample, namely the differential learning among multiple samples, and has important engineering application value.
Drawings
FIG. 1 is a flow chart of a model training phase of a sample relationship learning hyperspectral image classification method according to a specific embodiment of the invention.
FIG. 2 is a flowchart of a model application phase of a method for sample relationship learning hyperspectral image classification according to a specific embodiment of the invention.
Fig. 3 is a block diagram of a sample relationship learning hyperspectral image classification system according to a specific embodiment of the invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
The invention is characterized in that: extracting single sample features by adopting a deep neural network, and extracting common features of multiple samples by using the common features; then, by adopting feature linking, the features of the samples to be classified and the common features of the single classes are linked to obtain measurement features; finally, calculating the relation score of the metric characteristics by using metric learning; calculating a loss value in a training phase based on the calculated relation evaluation set based on the common characteristics of the samples to be classified and each class, updating parameters of a model by using a back propagation algorithm, judging class information of the samples to be classified through a maximum probability value during application, and generating a classification result graph pixel by pixel; the invention can fully excavate the difference between multi-class samples, and can realize high-precision identification of different classes of information in the hyperspectral image only by a small number of labeled samples.
Specifically, referring to fig. 1 and fig. 2, flowcharts of a training phase and an application phase of the method for classifying hyperspectral images through sample relationship learning according to the invention are shown.
A sample relation learning hyperspectral image classification method comprises
The method comprises two stages: a training method and an application method thereof,
the training method comprises the following specific steps:
target category sample set construction step S110: selecting a hyperspectral image with high quality and subjected to radiation correction from a hyperspectral image library of a certain sensor, and constructing a sample set X containing different target categories, wherein the number of each sample can be as small as 100 pixels at least;
a model support sample set construction step S120, wherein samples in the target category sample set X are divided into different subsets according to categories, namely, the samples with the same category are put together to form a single subset, and then the divided subsets are used as elements to form a new set to form a support sample set S;
a step S130 of constructing a model query sample set, in which the target category sample set X is copied and expanded in equal proportion, the quantity of each category sample after expansion is ensured to be the same, and a query sample set Q is obtained train For training;
a step S140 of constructing a model input example set, namely randomly selecting n samples S for each category in the support sample set S j ＝{x j1 ,…,x jn J e {1, \8230;, C }, the set of query samples Q used in training train In (1), randomly selecting a sample x q Composing a training input instance
And repeating the input instance construction process to obtain an input instance set for model training.
Single-sample feature extraction step S150: randomly selecting a number of instances from a training input instance set, for each instance
The samples in (1) comprise a support sample and a training query sample, and a deep neural network is applied
And respectively extracting the characteristics of each sample in the example to obtain the single sample characteristics of the supporting sample:
j ∈ {1, \8230;, C }, and the single-sample features of the training query sample
Wherein
Extracting parameters for the single sample features;
a common characteristic extraction step S160: will be described in detail with reference to the following examples
Support sample characteristics for each category in
Utilizing deep neural networks
Generating a commonality feature for the category
Wherein
Extracting parameters for the common features;
metric feature linking step S170: will now be described
The training query sample feature in (1), that is, the single sample feature of the sample to be classified
Features common to said classes
Obtaining characteristics to be measured by using link method
Wherein η is a characteristic linking parameter;
wherein the content of the first and second substances,
representing the query sample, i.e. the sample x to be classified q Is characterized in that
Showing the characteristics of the selected supporting sample of the j-th class
The metric features are features of the query sample
Characteristic common to supporting sample
Are connected.
Metric feature scoring step S180: characterizing said feature to be measured
Input to a deep neural network g ψ Obtaining a relationship score representing a training query sample x q Probability values belonging to the j-th class, where ψ is a measurement parameter;
model parameter updating step S190: calculating the loss sum of the selected multiple instances by using a loss function based on the relation score, updating the parameters of the model by using a back propagation mechanism, and finally finishing the training of the model, namely continuously iterating the steps S150-S180 according to the multiple instance sets, and optimizing the characteristic extraction parameters of the single sample
Common feature extraction parameter
The characteristic link parameter η and the metric parameter ψ until the loss function of an arbitrary batch instance set steadily approaches 0.
The application method comprises the following specific steps:
the method comprises the steps of sequentially deducing a category label for a given hyperspectral image to be classified according to a pixel-by-pixel method, constructing a sample example to be classified according to the pixel to be classified, sequentially applying trained related steps, including a single sample feature extraction step, a common feature extraction step, a measurement feature linking step and a measurement feature scoring step, obtaining the relationship scoring set of the sample example to be classified, then obtaining category information of the pixel to be classified according to maximum scoring, and finally generating a classification result image according to the pixel-by-pixel deduction result.
Optionally, the step S110 of constructing the target category sample set specifically includes:
defining C categories according to actual target types in the hyperspectral images, wherein the number of the categories is 1-C, C is a positive integer, respectively selecting Mi pixels as the center points of samples for the ith material category, extracting samples corresponding to the selected pixels according to specified rules including one-dimensional, two-dimensional or three-dimensional, and forming a target category sample set X by the selected target category samples;
the model support sample set constructing step S120 specifically includes:
dividing the target category sample set into C subsets according to categories S j J belongs to {1, \8230;, C }, and the support sample set of the model is obtained by taking the j as a set element
Optionally, the model query sample set constructing step S130 specifically includes:
copying and expanding the target category sample set according to a specified proportion to ensure that the number of samples in each category is consistent, ensuring that the categories of the samples are balanced in the training process, and obtaining a query sample set Q for model training train 。
Optionally, the step S140 of constructing a set of model input instances specifically includes:
in the supporting sample set S, each class randomly selects n samples to obtain the supporting sample S of each class j ＝{x j1 ,…,x jn J ∈ {1, \8230;, C }, and querying a sample set Q from the training train In the random selection of a sample x q The combination of selected sample components being an input instance
In the moldIn the type training phase, the marking strategy of the input example is as follows: if training the query sample x q Is given a label of y q Then input the example
Is also labeled as y q 。
According to the training input example marking strategy, the number of classes contained in the input example set in the training stage is C, the size of the target sample set X is assumed to be | | | X | |, and the number of samples contained in each class is the same as
The size of the constructed input instance set is
The requirement of the deep neural network on training can be met under the condition that the X is smaller.
In the training stage, according to the input instance construction method, a plurality of (B) input instances are constructed and used as training data of a model in a training batch, namely an input instance set of the training batch.
The single-sample feature extraction step S150 specifically includes:
by deep neural networks
As a single sample extraction, a batch is input into one instance of a set of instances
Respectively inputting the samples in (1) into a deep neural network
In (C.times.n) +1 single sample feature
Wherein
Is the single sample feature extraction parameter.
The common feature extracting step S160 specifically includes:
utilizing deep neural networks
Performing common feature extraction, and performing single-sample feature of the support sample of each category in the example
Input to a network
In the method, common characteristics are obtained
Wherein
For the common feature extraction parameters, for one input example, the common feature extraction will respectively obtain C common features, which represent class prototypes of C classes.
The metric feature linking step S170 specifically includes:
using a chaining function C η Common features of said certain classes
And training single sample features of the query sample
Linked to obtain metric features
Where η is the feature linking parameter.
The metric feature scoring step S180 specifically includes:
using deep neural networks g ψ A metric feature scoring module for scoring said metric features
Input to a deep neural network g ψ In (2), obtain a query sample x q And a plurality of supporting samples s j Is scored for similarity between the two samples
Wherein psi is a measurement parameter, and the relation evaluation set R q ＝[r q1 ,…,r qC ]Containing C values, r qj Representing training query samples x q With the selected supporting sample s j The similarity of the training query sample and the classification support sample represents the probability value of the j-th class, and then the loss value of the current input example is obtained by utilizing the loss function calculation according to the relation evaluation set, namely the relation evaluation between the training query sample and each classification support sample.
The model parameter updating step S190 includes:
selecting a plurality of examples from a certain batch, sequentially utilizing the steps S150-S180 to obtain the loss value of a single example, utilizing the loss function to calculate to obtain the loss value of the current input example, calculating the total loss of all the examples in the batch, and iteratively updating the parameters of the steps S150-S180 according to a back propagation method, wherein the updated parameters comprise single-sample feature extraction parameters
Common feature extraction parameters
A characteristic link parameter η and a metric parameter ψ; according to the batches of the multiple examples, continuously iterating the single-sample feature extraction parameters according to the example sets in the multiple batches
Common feature extraction parameters
Calculating the sum of loss values of input examples according to the characteristic link parameter eta and the measurement parameter psi, updating parameters according to the steps of integrating the bulk loss of batch examples and updating the parameters until the loss function of any example set is stably close to 0, and extracting parameters from the single sample characteristics after iteration is finished
Common feature extraction parameter
The combination of the characteristic link parameter η and the metric parameter ψ is a trained parameter.
Optionally, the specific step of generating the classified image includes:
after model training is completed, constructing a sample set to be classified pixel by pixel according to a certain rule for a hyperspectral image to be classified, and taking the sample set to be classified as a query sample set Q of an application stage apply And constructing a rule for the support sample set S in the training stage according to the input example to obtain the input example in the application stage
X 'here' q Is from Q apply Samples to be classified, s, selected in order in the set j J ∈ {1, \8230;, C } is n samples randomly selected from the jth class of support sample set; inputting the input example of the application stage
Extracting parameters using steps S150-S180 and trained single sample features
Common characteristicExtracting parameters
The characteristic link parameter eta and the metric parameter psi obtain a sample x 'to be classified' q Relationship score R with selected support sample q According to R q And determining the class label of the sample pair to be classified according to the position of the maximum probability value, deducing the class label pixel by pixel on the whole hyperspectral image to be classified, and generating a classification result image.
Optionally, in the model parameter updating step S190, a plurality of input instances selected from a certain batch are assumed to be B, the loss values of the single instances are obtained sequentially through steps S150 to S180, the loss value of the current input instance is obtained through calculation of a loss function, the total loss of all instances in the batch is calculated, and the parameters of S150 to S180 are iteratively updated according to a back propagation method, specifically:
the relationship evaluation set R calculated by each example is utilized q ＝[r q1 ,…,r qC ]Selecting a mean square error function as a loss function, wherein the optimization process is as shown in a formula:
wherein B represents the number of input instances in a certain batch, C represents the number of categories, y i Is supporting a sample s i Label of (a), y q Is a label for the query sample.
The formula shows that when y i ＝＝y q The loss function is calculated using the formula r qi -1, otherwise, the loss function uses r qi As a calculation term.
Further, referring to fig. 3, the invention also discloses a sample relation learning hyperspectral image classification system, which comprises
The device comprises two parts: a model training part and a model application part,
the model training part comprises the following concrete steps:
the target category sample set construction module 210: selecting a hyperspectral image with high quality and subjected to radiation correction from a hyperspectral image library of a certain sensor, and constructing a sample set X containing different target categories, wherein the number of each sample can be as small as 100 pixels at least;
the model support sample set construction module 220 divides the samples in the target category sample set X into different subsets according to categories, namely, the samples with the same category are put together to form a single subset, and then the divided subsets are used as elements to form a new set to form a support sample set S;
the model query sample set construction module 230 is to copy and expand the target category sample set X in equal proportion, so as to ensure the same number of each category sample after expansion, and obtain a query sample set Q train For training;
model input instance set construction module 240 for randomly selecting n samples S for each category in the support sample set S j ＝{x j1 ,…,x jn J ∈ {1, \ 8230;, C }, the set of query samples Q used in training train In (2), randomly selecting a sample x p Composing a training input instance
And repeating the input instance construction process to obtain an input instance set for model training.
Single sample feature extraction module 250: randomly selecting a number of instances from a training input instance set, for each instance
The samples in (1) comprise a support sample and a training query sample, and a deep neural network is applied
Respectively extracting the characteristics of each sample in the example to obtain the single-sample characteristics of the supporting sample:
j ∈ {1, \8230;, C }, and trainingSingle sample characterization of a query sample
Wherein
Extracting parameters for the single sample features;
the common feature extraction module 260: will now be described
Support sample characteristics for each category in
Utilizing deep neural networks
Generating common characteristics of the category
Wherein
Extracting parameters for the common features;
metric feature linking module 270: will be described in detail with reference to the following examples
The training query sample in (1), that is, the single-sample feature of the sample to be classified
Features common to said classes
Obtaining characteristics to be measured by using link method
Wherein η is a characteristic linking parameter;
metric feature scoring module 280: will be described inCharacteristics to be measured
Input to a deep neural network g ψ Obtaining a relationship score representing a training query sample x q Probability values belonging to class j, where ψ is a metric parameter;
model parameter update module 290: calculating the loss sum of the selected multiple instances by using a loss function based on the relation score, updating the parameters of the model by using a back propagation mechanism, and finally finishing the training of the model, namely continuously iterating the modules 250-280 according to the multiple instance sets to optimize the single-sample feature extraction parameters
Common feature extraction parameters
The feature linking parameter η and the metric parameter ψ are obtained until the loss function of any batch of the example set is stably approximated to 0, and a trained single-sample feature extraction module 250, a common feature extraction module 260 and a metric feature scoring module 280 are obtained.
That is, the single-sample feature extraction module 250, the common feature extraction module 260, and the metric feature scoring module 280 correspond to models that need to be trained.
The model application part comprises
The classification result image generation module 295: the method comprises the steps of classifying a given hyperspectral image to be classified in sequence according to a pixel-by-pixel method, constructing a sample example to be classified according to the pixel to be classified, obtaining the relation evaluation set of the sample example to be classified by sequentially applying trained modules, wherein the trained modules comprise a single sample feature extraction module 250, a common feature extraction module 260 and a measurement feature evaluation module 280, obtaining the category information of the pixel to be classified according to the maximum evaluation, and finally generating a classification result image according to the pixel-by-pixel inference result.
Example (b):
referring to fig. 2, a method for classifying hyperspectral images by using sample relationship learning according to the present invention is shown, which comprises the following steps:
a110, for the hyperspectral image to be classified, constructing a sample set to be classified according to a certain rule for each pixel on the image;
a120, taking the sample set to be classified as an application query sample set, and taking the support sample set in the training stage as a support sample set in the application stage;
a130, selecting support samples in the support sample set according to a certain rule, sequentially selecting query samples in the application query sample set, and forming an application input example by the query samples and the support samples;
a140, according to the application input example, obtaining a relation score R of the sample to be classified by using the trained model q ；
A150 according to the formula k Determining the label information of the sample to be classified at the position of the maximum probability value;
and A160, deducing category information pixel by pixel on the whole hyperspectral image to be classified to generate a classification result image.
In summary, the invention has the following advantages:
1. the depth model of the invention only needs a small amount of marking samples, each type can be less than 100 marking pixels, and high precision is achieved;
2. the invention can obtain a large amount of training data according to a small amount of samples, and the number of the marked samples is assumed to be | | X | | |, the marked samples comprise C categories, and the number of the samples contained in each category is the same as
The size of the constructed input instance set (training data set of the model) is then
3. The method can learn the relation score between the samples to be classified and each class sample, namely the differential learning among multiple samples, and has important engineering application value.
It will be apparent to those skilled in the art that the various elements or steps of the invention described above may be implemented using a general purpose computing device, they may be centralized on a single computing device, or alternatively, they may be implemented using program code that is executable by a computing device, such that they may be stored in a memory device and executed by a computing device, or they may be separately fabricated into various integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above is a further detailed description of the invention with reference to specific preferred embodiments, which should not be considered as limiting the invention to the specific embodiments described herein, but rather as a matter of simple deductions or substitutions by a person skilled in the art without departing from the inventive concept, it should be considered that the invention lies within the scope of protection defined by the claims as filed.
Claims (7)
1. A sample relation learning hyperspectral image classification method comprises
The method comprises two stages: a training method and an application method, wherein,
the training method comprises the following specific steps:
target class sample set construction step S110: selecting a hyperspectral image with high quality and subjected to radiation correction for a hyperspectral image library of a certain sensor, and constructing a sample set X containing different target categories, wherein the number of each sample can be as least as 100 pixels;
a model support sample set construction step S120: dividing the samples in the target category sample set X into different subsets according to categories, namely putting the samples with the same category together to form a single subset, and then taking the divided subsets as elements to form a new set to form a support sample set S;
model query sample set construction step S130: will be describedThe target category sample set X is copied and expanded according to equal proportion, the quantity of each category sample after expansion is ensured to be the same, and a query sample set Q is obtained train For training;
model input instance set construction step S140: in the supporting sample set S, n samples S are randomly selected for each category j ＝{x j1 ，…，x jn J e {1, \8230;, C }, the set of query samples Q used in training train In (2), randomly selecting a sample x q Composing a training input instance
Repeating the input instance construction process to obtain an input instance set of model training;
single sample feature extraction step S150: randomly selecting a number of instances from a training input instance set, for each instance
The sample in (1) comprises a support sample and a training query sample, and a deep neural network is applied
And respectively extracting the characteristics of each sample in the example to obtain the single sample characteristics of the supporting sample:
and training single sample features of the query sample
Wherein
Extracting parameters for the single sample features;
a common characteristic extraction step S160: will now be described
Support sample characteristics of each category in
Utilizing deep neural networks
Generating common characteristics of the category
Wherein
Extracting parameters for the common features;
metric feature linking step S170: will now be described
The training query sample feature in (1), that is, the single sample feature of the sample to be classified
Features common to said classes
Obtaining characteristics to be measured by using link method
Wherein η is a characteristic linking parameter;
metric feature scoring step S180: characterizing said feature to be measured
Input to a deep neural network g ψ Obtaining a relationship score representing a relationshipTraining query sample x q Probability values belonging to class j, where ψ is a metric parameter;
model parameter updating step S190: calculating the loss sum of the selected multiple examples by using a loss function based on the relation score, updating the parameters of the model by using a back propagation mechanism, and finally finishing the training of the model, namely continuously iterating the steps S150-S180 according to the multiple example sets, optimizing the single-sample characteristic and extracting the parameters
Common feature extraction parameter
The characteristic link parameter eta and the measurement parameter psi are obtained until the loss function of any batch of the example set is stably close to 0;
the application method comprises the following specific steps:
sequentially deducing category labels for a given hyperspectral image to be classified according to a pixel-by-pixel method, constructing a sample example to be classified according to the pixel to be classified, and sequentially applying trained related steps, wherein the trained related steps comprise a single sample feature extraction step, a common feature extraction step, a measurement feature linking step and a measurement feature scoring step, so as to obtain the relationship scoring set of the sample example to be classified, then obtaining the category information of the pixel to be classified according to the maximum scoring, and finally generating a classification result image according to the pixel-by-pixel deduction result.
2. The sample relationship learning hyperspectral image classification method according to claim 1, characterized in that:
the step S110 of constructing the target category sample set specifically includes:
defining C categories according to actual target types in the hyperspectral image, wherein the category numbers are 1-C, C is a positive integer, respectively selecting Mi pixels as the center points of samples for the ith material category, extracting samples corresponding to the selected pixels according to specified rules including one-dimensional, two-dimensional or three-dimensional, and forming a target category sample set X by the selected target category samples:
the model support sample set constructing step S120 specifically includes:
dividing the target category sample set into C subsets S according to categories j J ∈ {1, \8230;, C }, and the support sample set of the model is obtained by taking the j ∈ {1, \8230;, C } as a set element
The model query sample set constructing step S130 specifically includes:
copying and expanding the target category sample set according to a specified proportion to ensure that the number of samples in each category is consistent, ensuring that the categories of the samples are balanced in the training process, and obtaining a query sample set Q for model training train ；
The step S140 of constructing a set of model input instances specifically includes:
in the supporting sample set S, each class randomly selects n samples to obtain the supporting sample S of each class j ＝{x j1 ，…，x jn J ∈ {1, \8230;, C }, and querying a sample set Q from the training train In the random selection of a sample x q The combination of selected sample components being an input instance
In the model training phase, the marking strategy of the input example is as follows: if training the query sample x q Is given a label of y q Then input the example
Is also y q ；
According to the training input example marking strategy, the number of classes contained in the input example set in the training stage is C, the size of the target sample set X is assumed to be | | | X | |, and the number of samples contained in each class is the same as
The size of the constructed input instance set is
The requirement of the deep neural network on training can be met under the condition that the X is smaller.
3. The sample relationship learning hyperspectral image classification method according to claim 1 is characterized in that:
the single-sample feature extraction step S150 specifically includes:
by deep neural networks
As a single sample extraction, a batch is input into one instance of a set of instances
Respectively inputting the samples in (1) into a deep neural network
In (C.times.n) +1 single sample feature
Wherein
Is the single sample feature extraction parameter;
the common feature extracting step S160 specifically includes:
utilizing deep neural networks
Performing common feature extraction to list the supporting samples of each category in the exampleSample characterization
Input into a network
In the method, common characteristics are obtained
Wherein
Extracting parameters for the common features, wherein for one input example, the common feature extraction respectively obtains C common features which represent class prototypes of C classes;
the metric feature linking step S170 specifically includes:
using a chaining function C η Common features of said certain classes
And training single sample features of the query sample
Linked to obtain metric features
Wherein η is a characteristic linking parameter;
the metric feature scoring step S180 specifically includes:
using deep neural networks g ψ A metric feature scoring module for scoring said metric features
Input to a deep neural network g ψ In (2), obtain a query sample x q And a plurality of supporting samples s j Is scored for similarity between the two samples
Wherein psi is a measurement parameter, and the relation is evaluated as a set R q ＝[r q1 ，…，r qC ]Contains C values, r qj Representing training query samples x q With the selected supporting sample s j The similarity of the test data represents the probability value of the test data belonging to the jth class, and then the loss value of the current input example is obtained by utilizing the loss function calculation according to the relation evaluation set, namely the relation evaluation between the training query sample and each class support sample;
the model parameter updating step S190 includes:
selecting a plurality of instances of a certain batch, obtaining the loss value of a single instance by sequentially utilizing the steps S150-S180, obtaining the loss value of the current input instance by utilizing the loss function calculation, calculating the total loss of all the instances in the batch, and iteratively updating the parameters of the steps S150-S180 according to a back propagation method, wherein the updated parameters comprise single-sample feature extraction parameters
Common feature extraction parameter
A characteristic link parameter η and a metric parameter ψ; according to the batches of the multiple examples, continuously iterating the single-sample feature extraction parameters according to the example sets in the multiple batches
Common feature extraction parameters
Calculating the sum of loss values of input examples according to the characteristic link parameter eta and the metric parameter psi, and then aggregating the bulk loss and updating the parameters according to the batch examplesUpdating parameters until the loss function of any example set is stably close to 0, and extracting parameters from the characteristics of the single sample after iteration is finished
Common feature extraction parameters
The characteristic link parameter η and the metric parameter ψ are combined to be a trained parameter.
4. The sample relationship learning hyperspectral image classification method according to any one of claims 2 to 3 is characterized in that:
the specific steps of generating the classified image include:
after model training is completed, constructing a sample set to be classified pixel by pixel according to a certain rule by using a hyperspectral image to be classified, and taking the sample set to be classified as a query sample set Q of an application stage apply And constructing a rule for the support sample set S in the training stage according to the input example to obtain the input example in the application stage
X 'here' q Is from Q apply Samples to be classified, s, selected in order in the set j J ∈ {1, \8230;, C } is n samples randomly selected from the jth class of support sample set; inputting the input example of the application stage
Extracting parameters using steps S150-S180 and trained single sample features
Common feature extraction parameter
Feature linking parameterD, obtaining a sample x 'to be classified by the number eta and the metric parameter psi' q Relationship score R with selected support sample q According to R q And determining the class label of the sample pair to be classified according to the position of the maximum probability value, deducing the class label pixel by pixel on the whole hyperspectral image to be classified, and generating a classification result image.
5. The sample relationship learning hyperspectral image classification method according to claim 4, characterized in that:
in the model parameter updating step S190, a plurality of input instances B selected from a certain batch are sequentially subjected to steps S150 to S180 to obtain a loss value of a single instance, a loss function is used for calculation to obtain a loss value of a current input instance, the total loss of all instances in the batch is calculated, and the parameters of S150 to S180 are iteratively updated according to a back propagation method, specifically:
the relationship evaluation set R calculated by each example is utilized q ＝[r q1 ，…，r qC ]Selecting a mean square error function as a loss function, and optimizing the process according to a formula:
where B represents the number of input instances in a batch, C represents the number of categories, y i Is supporting a sample s i Label of (a), y q Is a label for the query sample.
6. A sample relation learning hyperspectral image classification system comprises
The device comprises two parts: a model training part and a model application part,
the model training part comprises the following concrete steps:
the target class sample set construction module 210: selecting a hyperspectral image with high quality and subjected to radiation correction for a hyperspectral image library of a certain sensor, and constructing a sample set X containing different target categories, wherein the number of each sample can be as least as 100 pixels;
model support sample set construction module 220: dividing the samples in the target category sample set X into different subsets according to categories, namely putting the samples with the same category together to form a single subset, and then taking the divided subsets as elements to form a new set to form a support sample set S;
model query sample set construction module 230: copying and expanding the target category sample set X according to equal proportion, ensuring that the quantity of each category sample after expansion is the same, and obtaining a query sample set Q train For training;
model input instance set construction module 240: in the supporting sample set S, n samples S are randomly selected for each category j ＝{x j1 ，…，x jn J ∈ {1, \ 8230;, C }, the set of query samples Q used in training train In (1), randomly selecting a sample x q Composing a training input instance
And repeating the input instance construction process to obtain an input instance set for model training.
Single sample feature extraction module 250: randomly selecting a number of instances from a training input instance set, for each instance
The sample in (1) comprises a support sample and a training query sample, and a deep neural network is applied
Extracting the characteristics of each sample in the example, respectively, supporting the single sample characteristics of the sample:
and training single sample features of the query sample
Wherein
Extracting parameters for the single sample features;
the common feature extraction module 260: will now be described
Support sample characteristics for each category in
Utilizing deep neural networks
Generating common characteristics of the category
Wherein
Extracting parameters for the common features;
metric feature linking module 270: will now be described
The training query sample in (1), i.e. the single sample feature of the sample to be classified
Features common to said class
Obtaining characteristics to be measured by using link method
Wherein η is a characteristic linking parameter;
metric feature scoring module 280: characterizing said feature to be measured
Input to a deep neural network g ψ Obtaining a relationship score representing a training query sample x q Probability values belonging to class j, where ψ is a metric parameter;
model parameter update module 290: calculating the loss sum of the selected multiple examples by using a loss function based on the relation score, updating the parameters of the model by using a back propagation mechanism, and finally finishing the training of the model, namely continuously iterating the modules 250-280 according to the multiple example sets to optimize the single-sample characteristic extraction parameters
Common feature extraction parameters
The feature linking parameter η and the metric parameter ψ are obtained until the loss function of any batch of the example set is stably approximated to 0, and a trained single-sample feature extraction module 250, a common feature extraction module 260 and a metric feature scoring module 280 are obtained.
7. The sample relationship learning hyperspectral image classification system according to claim 6, wherein:
the model application part comprises
The classification result image generation module 295: the method comprises the steps of classifying a given hyperspectral image to be classified in sequence according to a pixel-by-pixel method, constructing a sample example to be classified according to the pixel to be classified, obtaining the relation evaluation set of the sample example to be classified by sequentially applying trained modules, wherein the trained modules comprise a single sample feature extraction module 250, a common feature extraction module 260 and a measurement feature evaluation module 280, obtaining the category information of the pixel to be classified according to the maximum evaluation, and finally generating a classification result image according to the pixel-by-pixel inference result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011509345.1A CN112651317B (en) | 2020-12-18 | 2020-12-18 | Hyperspectral image classification method and system for sample relation learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011509345.1A CN112651317B (en) | 2020-12-18 | 2020-12-18 | Hyperspectral image classification method and system for sample relation learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112651317A CN112651317A (en) | 2021-04-13 |
| CN112651317B true CN112651317B (en) | 2022-11-25 |
Family
ID=75355340
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011509345.1A Active CN112651317B (en) | 2020-12-18 | 2020-12-18 | Hyperspectral image classification method and system for sample relation learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112651317B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113128588B (en) * | 2021-04-16 | 2024-03-26 | 深圳市腾讯网域计算机网络有限公司 | Model training method, device, computer equipment and computer storage medium |
| CN113221964B (en) * | 2021-04-22 | 2022-06-24 | 华南师范大学 | Single sample image classification method, system, computer device and storage medium |
| CN113378985A (en) * | 2021-07-06 | 2021-09-10 | 浙江工业大学 | Countermeasure sample detection method and device based on layer-by-layer correlation propagation |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110852227A (en) * | 2019-11-04 | 2020-02-28 | 中国科学院遥感与数字地球研究所 | Hyperspectral image deep learning classification method, device, equipment and storage medium |
| CN111401426A (en) * | 2020-03-11 | 2020-07-10 | 西北工业大学 | Small sample hyperspectral image classification method based on pseudo label learning |
-
2020
- 2020-12-18 CN CN202011509345.1A patent/CN112651317B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110852227A (en) * | 2019-11-04 | 2020-02-28 | 中国科学院遥感与数字地球研究所 | Hyperspectral image deep learning classification method, device, equipment and storage medium |
| CN111401426A (en) * | 2020-03-11 | 2020-07-10 | 西北工业大学 | Small sample hyperspectral image classification method based on pseudo label learning |
Non-Patent Citations (2)
| Title |
|---|
| 基于卷积神经网络与主动学习的高光谱图像分类;宋晗等;《中国科学院大学学报》;20200315(第02期);全文 * |
| 高光谱影像分类的深度少样例学习方法;刘冰等;《测绘学报》;20201015(第10期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112651317A (en) | 2021-04-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112651317B (en) | Hyperspectral image classification method and system for sample relation learning | |
| D’Isanto et al. | Photometric redshift estimation via deep learning-generalized and pre-classification-less, image based, fully probabilistic redshifts | |
| CN108052911B (en) | Deep learning-based multi-mode remote sensing image high-level feature fusion classification method | |
| Zortea et al. | Spatial preprocessing for endmember extraction | |
| US20190318256A1 (en) | Method, apparatus and system for estimating causality among observed variables | |
| CN110619059B (en) | Building marking method based on transfer learning | |
| CN111783884B (en) | Unsupervised hyperspectral image classification method based on deep learning | |
| Kuntzer et al. | Stellar classification from single-band imaging using machine learning | |
| CN105095913A (en) | Remote sensing image classification method and system based on neighbor regular joint sparse representation | |
| CN113705641A (en) | Hyperspectral image classification method based on rich context network | |
| CN110310012B (en) | Data analysis method, device, equipment and computer readable storage medium | |
| CN116129219A (en) | SAR target class increment recognition method based on knowledge robust-rebalancing network | |
| CN113516019B (en) | Hyperspectral image unmixing method and device and electronic equipment | |
| Dovbysh et al. | Estimation of Informativeness of Recognition Signs at Extreme Information Machine Learning of Knowledge Control System. | |
| CN111222576B (en) | High-resolution remote sensing image classification method | |
| CN114556364A (en) | Neural architecture search based on similarity operator ordering | |
| Padhy et al. | Image classification in artificial neural network using fractal dimension | |
| CN111782805A (en) | Text label classification method and system | |
| TW569145B (en) | Pattern classifying method and device thereof, and computer readable record medium | |
| Hechinger et al. | Categorising the world into local climate zones: towards quantifying labelling uncertainty for machine learning models | |
| CN110059606A (en) | A kind of improved increment Non-negative Matrix Factorization face recognition algorithms | |
| Ososkov et al. | Simulation and Analysis of the Properties of Linear Structures in the Mass Distribution of Nuclear Reaction Products by Machine Learning Methods | |
| CN111091087B (en) | Land coverage extraction algorithm based on multi-view collaborative canonical correlation forest remote sensing image | |
| MS | A Hand Written Digit Recognition Using Convolutional Neural Networks Compared With Decision Tree With Improved Accuracy. | |
| US20230419086A1 (en) | System and method for processing material properties of structural materials |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |