CN117593649A - Unbalanced hyperspectral image integrated classification method, unbalanced hyperspectral image integrated classification system and electronic equipment - Google Patents

Abstract

The invention discloses an unbalanced hyperspectral image integrated classification method, an unbalanced hyperspectral image integrated classification system and electronic equipment, and relates to the technical field of image processing. According to the invention, spectrum-space feature fusion is carried out firstly, spatial information is fused into the spectrum of a sample, then sample synthesis is carried out on a few samples in the original hyperspectral image data set according to a set sampling strategy in the fused spectrum-space feature space, a balanced sample set is obtained, and optimization of the sample space is realized; and solving a plurality of feature subset projection matrixes obtained by multiple random division by using a Fisher discriminant analysis algorithm, constructing a plurality of corresponding sparse rotation matrixes by using a transformed projection vector feature alignment method, respectively inputting the obtained plurality of rotation sample sets into corresponding support vector machine base classifiers, and integrating the base classifiers by adopting D-S fusion to determine classification results, thereby realizing optimization of a decision space.

Description

Unbalanced hyperspectral image integrated classification method, unbalanced hyperspectral image integrated classification system and electronic equipment

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an unbalanced hyperspectral image integrated classification method, system, and electronic device.

Background

The hyperspectral image records hundreds of narrow-band spectrum information of an observation target, can provide rich high-dimensional characteristics for subsequent image processing tasks, and is widely used in tasks of distinguishing different ground objects or materials, detecting environmental changes and the like. Because the observed features or other objects differ in number, size, and spatial distribution, hyperspectral images are characterized by an imbalance, which is manifested by a significantly higher proportion of partial class samples than other classes. Most standard learning algorithms tend to favor most classes of samples, while sacrificing classification accuracy for a few classes, in order to optimize overall classification errors. This results in some excellent algorithms that, while achieving relatively high overall accuracy in processing unbalanced data sets, perform poorly in the identification of some minority categories. In practical applications, however, people often pay more attention to accurate recognition of minority classes. For example, for detection of a particular object, the object occupies far fewer pixels than land, trees, etc. in the background, and correctly identifying the pixels that contain the object is more important than correctly identifying the pixels of other features. Therefore, improvement of the classification method is needed for the data unbalance characteristic of hyperspectrum.

Aiming at the problem of unbalanced hyperspectral image classification, the current research mainly combines resampling to conduct class rebalancing, for example, feng Wei of the university of western electronic technology, etc., firstly, a principal component analysis method is used for carrying out dimension reduction on an original hyperspectral image, and a minority of samples after dimension reduction are oversampled; zhong Xian, et al, university of western electronics, put the synthetic oversampled data into a multi-layer deep random forest. Chu Heng, et al, of Chongqing university of post and telecommunications, propose a hyperspectral image imbalance classification method based on mean shift and oversampling, and perform maximum voting fusion on a segmentation map obtained based on a mean shift algorithm and a classification map obtained based on a Smote algorithm. Tong Yingping et al, of the university of western electronics and technology, designed an unbalanced integrated classification algorithm using dynamic sampling factor techniques, did not perform feature space optimization for unbalanced data, although rotation of feature space was performed using a rotating forest, and used a simple voting decision fusion method for rotating forests. Yuan Peisen et al of Nanjing university of agriculture propose a classification method based on SVC (support vector classification ) and oversampling, and after rebalancing of unbalanced agricultural hyperspectral data, the SVC algorithm is utilized to realize multi-classification based on a support vector machine SVM through voting fusion.

Based on the above description, the existing method mainly focuses on optimizing a sample space by using a synthetic oversampling algorithm, does not consider optimizing from a feature space, finds a feature space more suitable for characterizing unbalanced data, and classifies based on the optimized feature space. In addition, for the unbalanced hyperspectral method requiring decision fusion, simple voting fusion is often adopted, and optimization of decision space is not further considered.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides an unbalanced hyperspectral image integrated classification method, an unbalanced hyperspectral image integrated classification system and electronic equipment.

In order to achieve the above object, the present invention provides the following solutions:

an unbalanced hyperspectral image integrated classification method, comprising:

performing sample synthesis on a few samples in the original hyperspectral image data set according to a set sampling strategy in the fused spectrum-space characteristic space based on a spectrum-space neighborhood self-adaptive artificial synthesis oversampling algorithm to obtain a balanced sample set; the minority samples are samples with the class unbalance degree lower than a set value;

randomly dividing the characteristics in the balance sample set for a plurality of times to obtain a plurality of characteristic subsets;

solving a projection matrix of the feature subset obtained by random division through a Fisher discriminant analysis algorithm, and constructing a plurality of corresponding rotation matrices by using a reconstructed projection vector feature alignment method;

performing feature space transformation on the data matrix of the balanced sample set based on a plurality of rotation matrices to obtain a plurality of rotation sample sets;

and correspondingly inputting the rotation sample sets on each different feature space into a support vector machine base classifier, and integrating all support vector machine base classifiers by using D-S fusion to obtain classification results so as to finish decision space optimization.

Optionally, based on a spectrum-space neighborhood adaptive artificial synthesis oversampling algorithm, sample synthesis is performed on a few samples in an original hyperspectral image dataset according to a set sampling strategy in a fused spectrum-space feature space to obtain a balanced sample set, which specifically comprises:

scanning a two-dimensional image in an original hyperspectral image dataset by using a sliding window to obtain a track matrix, and embedding space structure information into the track matrix to finish the fusion of spectrum-space characteristic space; the track matrix has a hanker-block-hanker structure, expressed as a hanker matrix with respect to the sub-matrix;

singular value decomposition is carried out on the basis of the track matrixes to obtain component matrixes, and the contribution of each component matrix to carrying track matrix information is determined;

grouping the component matrixes according to the contribution to obtain a grouping matrix;

recovering the Hank structure of each sub-matrix in the grouping matrix through the diagonal average operation of the auxiliary diagonal, recovering the Hank structure of each matrix formed by the sub-matrices recovering the Hank structure through the diagonal average operation of the auxiliary diagonal based on the sub-matrix, and completing two-step Hank processing to obtain a Hank-block-Hank matrix;

reconstructing two-dimensional signals based on the Hank-block-Hank matrix to obtain a reconstructed two-dimensional image matrix;

and carrying out sample resampling based on the reconstructed two-dimensional image matrix to obtain a balanced sample set.

Optionally, a sliding window is used for scanning a two-dimensional image in the original hyperspectral image dataset to obtain a track matrix, so that the space structure information is embedded into the track matrix, and the method specifically comprises the following steps:

scanning a two-dimensional image in the original hyperspectral image dataset by using a sliding window to obtain a window matrix;

and vectorizing the window matrix and splicing the window matrix row by row to obtain the track matrix embedded with the space structure information.

Optionally, performing singular value decomposition based on the track matrix to obtain component matrices, and determining a contribution of each component matrix to carrying track matrix information, which specifically includes:

obtaining a matrix product, and carrying out eigenvalue decomposition on the matrix product to obtain an eigenvalue and an eigenvector corresponding to the eigenvalue; the matrix product is the product of the trajectory matrix and a transpose matrix of the trajectory matrix;

determining a component matrix based on the eigenvalues and eigenvectors corresponding to the eigenvalues;

and taking the eigenvalue corresponding to the component matrix as the contribution.

Optionally, grouping the component matrices according to the contribution amounts to obtain grouping matrices, which specifically include:

dividing the component matrix into a plurality of disjoint subsets;

and reserving a subset with the contribution quantity exceeding a set value in the plurality of disjoint subsets to obtain the grouping matrix.

Optionally, reconstructing the two-dimensional signal based on the hanker-block-hanker matrix to obtain a reconstructed two-dimensional image matrix, which specifically includes:

vectorizing each Hanker-block-Hanker matrix to obtain a vector;

and splicing the vectors in the row direction according to the auxiliary diagonal order to obtain a reconstructed two-dimensional image matrix.

Optionally, performing sample resampling based on the reconstructed two-dimensional image matrix to obtain a balanced sample set, which specifically includes:

determining a pixel sample vector based on the reconstructed two-dimensional image matrix;

determining the sample number of a certain class based on the class corresponding to the pixel sample vector;

determining a certain class of unbalance based on the number of samples;

when the unbalance is smaller than a balance threshold, setting an oversampling rate, and determining the number of samples to be synthesized in the category based on the oversampling rate;

determining a characteristic neighborhood of a sample by adopting a K Nearest Neighbor (KNN) algorithm, and determining a space neighborhood according to a pixel space position to obtain a spectrum-space neighborhood of the sample;

determining the heterogeneous sample proportion of the sample in the spectrum-space neighborhood;

normalizing all samples in the category based on the heterogeneous sample proportion to obtain sampling probability that the samples in the category are selected as seed samples;

determining the number of synthesized samples using the samples belonging to the category as seed samples according to the sampling probability and the number of the samples to be synthesized;

and carrying out repeated random neighbor extraction and interpolation treatment on the seed samples according to the number of the synthesized samples to obtain synthesized samples of the seed samples so as to obtain the balanced sample set.

Optionally, solving a projection matrix of the feature subset through a Fisher discriminant analysis algorithm, and constructing a rotation matrix by using a modified projection vector feature alignment method, wherein the method specifically comprises the following steps of:

constructing an all-zero matrix;

randomly extracting samples with set proportions from the characteristic subsets with places to obtain a training set;

solving a sample intra-class hash matrix and a sample inter-class hash matrix on a subspace for each feature subset;

determining all eigenvalues of a square matrix and eigenvectors corresponding to the eigenvalues based on the intra-sample-class hash matrix and the inter-sample-class hash matrix according to FDA criteria;

sorting the feature vectors according to the feature values to obtain a projection matrix;

determining a feature corpus index corresponding to the features in the projection matrix;

and placing the features in the projection matrix at the corresponding positions of the all-zero matrix according to the feature corpus index to obtain the rotation matrix.

According to the specific embodiment provided by the invention, the invention discloses the following technical effects:

aiming at the problem of unbalanced hyperspectral image classification, the invention respectively develops optimization research in three dimensions of a sample space, a feature space and a decision space so as to enhance the robustness of the classifier for unbalanced data, reduce the performance difference among classes and improve the overall classification performance. Aiming at the problem that the prior method ignores space information in the resampling process of sample space optimization, the invention firstly carries out spectrum-space feature fusion, fuses the space information into the spectrum of a sample, then carries out sample synthesis on a few samples in the original hyperspectral image data set according to a set sampling strategy in the fused spectrum-space feature space to obtain a balanced sample set so as to carry out sample space optimization; aiming at the problem that the existing method does not pay attention to feature space optimization for unbalanced data representation, the invention solves a projection matrix of a feature subset by using Fisher discriminant analysis (Fisher Discriminant Analysis, FDA) algorithm, and builds a sparse rotation matrix by using a transformed projection vector feature alignment method to finish feature space optimization; finally, aiming at the problem that the prior method generally adopts simplified voting in the decision fusion process without optimizing the decision space, the invention respectively inputs a plurality of rotary sample sets into corresponding support vector machine base classifiers, and integrates and determines classification results on the base classifiers by adopting D-S fusion, thereby completing the optimization of the decision space.

Further, the invention provides an unbalanced hyperspectral image integrated classification system for implementing the unbalanced hyperspectral image integrated classification method; the system comprises:

the sample space optimization module is used for carrying out sample synthesis on a few samples in the original hyperspectral image data set according to a set sampling strategy in the fused spectrum-space characteristic space based on a self-adaptive artificial synthesis oversampling algorithm of the spectrum-space neighborhood to obtain a balanced sample set; the minority samples are samples with the class unbalance degree lower than a set value;

the feature space optimization module is used for carrying out multiple random division on the features in the balance sample set to obtain a plurality of feature subsets, solving a projection matrix of the feature subsets obtained by the random division through a Fisher discriminant analysis algorithm, constructing a plurality of corresponding rotation matrixes by using a modified projection vector feature alignment method, and carrying out feature space transformation on a data matrix of the balance sample set based on the rotation matrixes to obtain a plurality of rotation sample sets;

and the decision space optimization module is used for correspondingly inputting the rotation sample sets on each different feature space into a support vector machine base classifier, and integrating all support vector machine base classifiers by using D-S fusion to obtain classification results so as to finish decision space optimization.

Still further, the present invention also provides an electronic device including:

a memory for storing a computer program;

and the processor is connected with the memory and is used for calling and executing the computer program so as to implement the unbalanced hyperspectral image integrated classification method.

The technical effects achieved by the system and the electronic device provided by the invention are the same as those achieved by the unbalanced hyperspectral image integrated classification method provided by the invention, so that the detailed description is omitted.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of an unbalanced hyperspectral image integrated classification method provided by the invention;

fig. 2 is a schematic diagram of an implementation of the unbalanced hyperspectral image integrated classification system provided by the invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The invention aims to provide an unbalanced hyperspectral image integrated classification method, an unbalanced hyperspectral image integrated classification system and electronic equipment, which can optimize a sample space, a feature space and a decision space so as to enhance the robustness of a classifier for unbalanced data, reduce the performance difference among classes and improve the overall classification performance.

In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.

As shown in fig. 1, the unbalanced hyperspectral image integrated classification method of the present invention comprises:

step 100: and (3) carrying out sample synthesis on a few samples in the original hyperspectral image data set according to a set sampling strategy in the fused spectrum-space characteristic space based on a spectrum-space neighborhood self-adaptive artificial synthesis oversampling algorithm to obtain a balanced sample set. The minority samples are samples with the class unbalance below a set value.

In the practical application process, the implementation process of the step can be as follows:

step 1, scanning a two-dimensional image in an original hyperspectral image dataset by using a sliding window to obtain a track matrix, and embedding space structure information into the track matrix to finish the fusion of a spectrum-space characteristic space. Specifically, the implementation process of the step 1 includes:

and step 1-1, scanning a two-dimensional image in the original hyperspectral image dataset by using a sliding window to obtain a window matrix.

Step 1-2, vectorizing the window matrix and splicing row by row to obtain a track matrix embedded with space structure information, wherein the track matrix of the two-dimensional signal has a Hanker-block-Hanker structure and can be expressed as a Hanker matrix related to the submatrix.

And 2, carrying out singular value decomposition based on the track matrix to obtain component matrices, and determining the contribution of each component matrix to carrying track matrix information. Specifically, the implementation process of the step 2 includes:

and 2-1, obtaining a matrix product, and carrying out eigenvalue decomposition on the matrix product to obtain eigenvalues and eigenvectors corresponding to the eigenvalues. The matrix product is the product of the trajectory matrix and the transpose of the trajectory matrix.

And 2-2, determining a component matrix based on the eigenvalues and eigenvectors corresponding to the eigenvalues.

And 2-3, taking the characteristic value corresponding to the component matrix as a contribution.

And step 3, grouping the component matrixes according to the contribution quantity to obtain a grouping matrix. Specifically, the implementation process of the step 3 includes:

step 3-1, dividing the component matrix into a plurality of disjoint subsets.

And 3-2, reserving a subset with contribution exceeding a set value in a plurality of disjoint subsets to obtain a grouping matrix.

Step 4, recovering the Hank structure of each sub-matrix in the grouping matrix through the diagonal average operation of the auxiliary diagonal, and recovering the Hank structure of each matrix formed by the sub-matrices recovering the Hank structure through the diagonal average operation of the auxiliary diagonal based on the sub-matrix, thereby completing two-step Hank-block-Hank matrix processing;

and 5, reconstructing two-dimensional signals based on the Hank-block-Hank matrix to obtain a reconstructed two-dimensional image matrix. Specifically, the implementation process of step 5 includes:

and 5-1, vectorizing each Hanker-block-Hanker matrix to obtain vectors.

And 5-2, splicing the vectors in the row direction according to the auxiliary diagonal order to obtain a reconstructed two-dimensional image matrix.

And 6, carrying out sample resampling based on the reconstructed two-dimensional image matrix to obtain a balanced sample set. Specifically, the implementation process of step 6 includes:

and 6-1, determining pixel sample vectors based on the reconstructed two-dimensional image matrix.

And 6-2, determining the sample number of a certain class based on the class corresponding to the pixel sample vector.

And 6-3, determining the unbalance degree of a certain category based on the number of the samples.

And 6-4, setting an oversampling rate when the unbalance degree is smaller than the balance degree threshold value, and determining the number of samples to be synthesized in the category based on the oversampling rate.

And 6-5, determining a characteristic neighborhood of the sample by adopting a K Nearest Neighbor (KNN) algorithm, and determining a spatial neighborhood according to the spatial position of the pixel to obtain a spectrum-empty neighborhood of the sample.

And 6-6, determining the heterogeneous sample proportion of the sample in the spectrum-space neighborhood.

And 6-7, normalizing all samples in the category based on the heterogeneous sample proportion to obtain the sampling probability that the samples in the category are selected as seed samples.

And 6-8, determining the number of synthesized samples using the samples belonging to the category as seed samples according to the sampling probability and the number of the samples to be synthesized.

And 6-9, repeating random neighbor extraction and interpolation processing on the seed samples according to the number of the synthesized samples to obtain synthesized samples of the seed samples so as to obtain a balanced sample set.

Step 101: and randomly dividing the features in the balanced sample set for a plurality of times to obtain a plurality of feature subsets.

Step 102: and solving a projection matrix of the feature subset obtained by random division through a Fisher discriminant analysis algorithm, and constructing a plurality of corresponding rotation matrices by using a modified projection vector feature alignment method.

In the practical application process, the implementation process of step 102 may be:

and 1, constructing an all-zero matrix.

And step 2, randomly extracting samples with set proportions from the characteristic subsets in a place-of-return mode to obtain a training set.

And 3, solving a sample intra-class hash matrix and a sample inter-class hash matrix on the subspace for each feature subset.

And 4, determining all eigenvalues and eigenvectors corresponding to the eigenvalues of the square matrix based on the hash matrix in the sample class and the hash matrix among the sample classes according to the FDA criterion.

And 5, sorting the eigenvectors according to the eigenvalues to obtain a projection matrix.

And 6, determining the feature corpus index corresponding to the features in the projection matrix.

And 7, placing the features in the projection matrix at the corresponding positions of the all-zero matrix according to the feature corpus index to obtain a rotation matrix.

Step 103: and performing feature space transformation on the data matrix of the balanced sample set based on the plurality of rotation matrices to obtain a plurality of rotation sample sets.

Step 104: and correspondingly inputting the rotation sample sets on each different feature space into a support vector machine base classifier, and integrating all support vector machine base classifiers by using D-S fusion to obtain classification results so as to finish decision space optimization.

Further, the invention provides an unbalanced hyperspectral image integrated classification system which is used for implementing the unbalanced hyperspectral image integrated classification method. The system comprises: the system comprises a sample space optimization module, a feature space optimization module and a decision space optimization module.

The sample space optimization module is used for carrying out sample synthesis on a few samples in the original hyperspectral image data set according to a set sampling strategy in the fused spectrum-space characteristic space based on a self-adaptive artificial synthesis oversampling algorithm of the spectrum-space neighborhood, so as to obtain a balanced sample set. The minority samples are samples with the class unbalance below a set value.

The feature space optimization module is used for carrying out multiple random division on the features in the balance sample set to obtain a plurality of feature subsets, solving a projection matrix of the feature subsets obtained by the random division through a Fisher discriminant analysis algorithm, constructing a plurality of corresponding rotation matrixes by using a modified projection vector feature alignment method, and carrying out feature space transformation on a data matrix of the balance sample set based on the rotation matrixes to obtain a plurality of rotation sample sets;

and the decision space optimization module is used for correspondingly inputting the rotation sample sets on each different feature space into a support vector machine base classifier, and integrating all support vector machine base classifiers by using D-S fusion to obtain classification results so as to finish decision space optimization.

In the practical application process, the sample space optimization module is divided into two stages of feature fusion and resampling. The method mainly uses a spectrum-space neighborhood-based self-adaptive synthesis oversampling algorithm to carry out sample synthesis oversampling on a minority class in an original training set (namely an original hyperspectral image dataset) in a fused spectrum-space characteristic space according to a set sampling strategy to obtain a balanced sample set. Specific:

(1) Spectral-spatial feature fusion. The spectrum-space feature fusion consists of four steps of embedding, singular value decomposition, grouping and reconstruction. For example, let the size of the hyperspectral image beThe two-dimensional image over a certain band may be represented as an image matrix.

The embedding step refers to scanning a two-dimensional image using a sliding window and embedding spatial structure information extracted from an original image matrix into a track matrix. Specifically, the window size is defined asWherein, the method comprises the steps of, wherein,. Scanning a two-dimensional image from left to right, top to bottom using a window +.>。/>Representing the spectral values. When the upper left corner gives a reference point +.>In this case, the window information obtained by scanning can be expressed as a window matrix +.>：

。

Using operatorsRepresenting vectorization of the matrix, i.e. row-by-row splicing of the matrix, the track matrix +.>Can be expressed as:

。

the track matrix of the two-dimensional signal has a Hankel-block-Hankel (HBH) structure, i.e. the track matrixCan be expressed as a hanker matrix with respect to the sub-matrices, with each sub-matrix +.>But also conforms to the Hank structure. The locus matrix is represented by adopting a Hanker structure, and is as follows:

。

。

is a sub-matrix element.

The singular value decomposition step is implemented as follows: pair matrixDecomposing the characteristic value, and calculating to obtain characteristic values of the arrangement from big to small +.>The corresponding feature vectors are +.>. Wherein (1)>Representing the track matrix +.>Is able to write out the track matrix +.>Defining each term in the singular value expansion as a component matrix +.>：

。

Is of intermediate quantity->。

Each component matrixTrack matrix->The contribution amount (amount of information carried) of (a) can be determined by the corresponding characteristic value +.>Measurement, upper corner mark->Representing the transpose of the matrix.

The grouping step is to decompose SVDThe individual component matrix is divided into->A subset of the non-intersecting parts->，. Defining a grouping matrix->：

。

The contribution of each group may be measured by the sum of the corresponding eigenvalues of the components within the group. The method comprises the steps of extracting main space structure information and achieving the effect of noise filtering, and grouping by adopting a setting mode of reserving groups with larger contributions.

The reconstruction step is to make the matrix obtain the HBH characteristic again through a two-step Han formulation (Hankelization) step under the condition that the selected grouping matrix is not necessarily still provided with the HBH structure after being combined, so that the two-dimensional signal can be reconstructed. Specific: first, the Hank structure is restored by diagonal averaging operation of the sub-diagonals for each sub-matrix, and then, in the sub-matrixEach sub-matrix is Hamming within the group matrix. Wherein, the sub-diagonals of each sub-matrix are numbered from top left to bottom right, usingRepresenting the +.>Elements on the bar sub diagonal, +.>Representing elements on the minor diagonal, then p ∈ ->(i.e.)>) Matrix->The process of performing the Hamming is defined as follows:

。

in the second-order regularization process, diagonal averages about the secondary diagonals, i.e., averages between corresponding elements, are still performed as above. And reconstructing a two-dimensional signal by using the obtained Hank matrix. The reconstruction process is as follows: firstly vectorizing each submatrix, then splicing the obtained row vectors in the row direction according to the auxiliary diagonal order, and obtaining a reconstructed newA two-dimensional image matrix.

(2) Resampling. Resampling consists of two steps, calculating the sampling probability and generating the composite sample.

The implementation process for calculating the sampling probability comprises the following steps: let pixel sample vectorCorresponding categoryNumbered as，/>Representing the total number of categories of the multi-category. Defining the class with the largest number of samples as the majority class +.>The rest are minority groups->Use +.>Representation category->The number of samples in (1) is->Representation category->The sample in (a) category->Imbalance of +.>Can be defined by the following formula:

。

wherein,. Setting a threshold value for the tolerable imbalance>When->Description category->Is higher, an oversampling is required, and an oversampling rate is set +.>For controlling in category->The number of samples synthesized in->The method comprises the following steps:

。

when (when)When, i.e. the number of classes with incomplete balance between all classes is to be oversampled, +.>It means that a perfectly balanced data set will be generated after oversampling. Calculating a pixel sample vector using the Euclidean distance of the feature>Is->Personal feature neighbor->Representing intra-feature neighborhood and pixel sample vector +.>The number of samples of different categories. Use->Selecting a pixel sample vector +.>Spatially->Nearest neighbor->Representing the number of spatial neighbors in the training set, +.>Representing spatial intra-neighborhood and->The number of training samples of different categories.

For categoryWhen oversampling is performed, it is necessary to first calculate the pixel sample vector +.>Heterogeneous sample ratio in spectral-spatial neighborhood ≡>Then normalize all samples in the class to obtain a pixel sample vector +.>Sampling probability selected as seed sample +.>The method comprises the following steps:

。

。

further, the system selects standard SMOTE algorithm to generate synthetic sample, and selects seed sampleOn the basis of (1) randomly selecting a homogeneous sample in its spectral-empty neighborhood +.>And uses the random value +.>Performing random interpolation to obtain a synthesized sample +.>The method comprises the following steps:

。

from sampling probabilityCan calculate that the usage belongs to the category +.>Pixel sample vector +.>The number of synthetic samples to be generated as seed sample +.>The method comprises the following steps:

。

repeatingSub-random neighbor extraction and interpolation can be performed to complete a seed sample (i.e., pixel sample vector +.>) Is a manual sample synthesis of (a).

Further, in the practical application process, the feature space optimization module is composed of two stages of rotation matrix construction and rotation sample set reconstruction.

(1) And (5) constructing a rotation matrix.

The method aims to solve the problem of class overlapping of unbalanced data by a feature space optimization method, reduce the difficulty of unbalanced classification and further solve the problem of low performance of few classes. And meanwhile, different data inputs are provided for the basic classifier in the subsequent integration stage by searching diversified spaces. To accomplish the objective, firstly, random feature subset division is performed on the features in the balanced sample set, then, creatively proposes to solve the problem of similar overlapping by combining feature extraction, namely, solving a projection matrix of the feature subset through Fisher discriminant analysis (Fisher Discriminant Analysis, FDA) algorithm, and constructing a sparse rotation matrix by using a modified projection vector feature alignment methodThe original dataset may be transformed in feature space by a rotation matrix. Thus, diversity can be introduced for the integrated classifier by multiple random subset divisions.

Wherein the rotation matrix is divided randomly for each timeThe specific steps for carrying out the solution are as follows:

(1) will rotate the matrixInitialized to->All zero matrices of (a).

(2) 75% of training samples are randomly extracted with a place of return, and a new training set is obtained.

(3) For each feature subset, solving for the intra-sample-class hash matrix over the subspaceAnd sample inter-class hash matrix>. Wherein, the hash matrix in the sample class +.>And summing various covariance matrixes in the hyperspectral image. Sample inter-class hash matrix>The method is obtained by calculating the distances between the center points of various samples and the center points of all samples. The feature space formed by the feature subsets is a subspace.

According to the FDA guidelines,for matrix->Is described.

(4) Solving square matrixIs to the feature vector from the higher to the lower feature value>Sequencing to obtain a feature vector sequence +.>The feature matrix consisting of it>The method comprises the following steps:

。/>is a feature vector sequence element.

(5) Feature alignment and feature subset recordingThe feature corpus index corresponding to each feature is +.>Then the element->Placed in matrix->Is>Line, th->The columns are:

。

(6) repeating the steps (2) - (4) for all the sub-divided feature subsets to obtain a rotation matrix。

(2) The rotated sample set is reconstructed.

Set the firstThe rotation matrix obtained by sub-division is +.>Data matrix for sample space optimized sample setPerforming feature space conversion to obtain balanced rotation sample set +.>The method comprises the following steps:

。

correspondingly, the balanced rotated sample set can be expressed as a rotated training set。

Furthermore, in the practical application process, in order to further improve the classification overall performance of the hyperspectral image, after passing through the space optimization module and the feature space optimization module, the decision layer fusion is performed by using the decision space optimization module. The decision space optimization module consists of three steps of classifier performance evaluation, confidence function generation and multi-classifier fusion.

The performance evaluation process of the multi-classifier comprises the following steps: assume a commonA plurality of SVM-based classifiers, each based classifierThere is confusion matrix->：/>

。

Wherein,indicate->Will be->Class image classification as +.>To eliminate the dimension influence, the elements in the confusion rule are normalized to：

。/>Is the element after normalization.

Obtaining a new confusion matrix with classification probability as：

。

At this time, the firstIndividual classifier pair category->The classification performance of (2) is->：

。

Further, the posterior classification probability of each classifier is calculatedIn combination with the classification property of the last step, a classification property of the first order is obtained>The category of->Basic confidence function of class->：

。

And the formula satisfies the condition。

Further, the multi-classifier fusion process is as follows:

assume that there is an identification framework，/>，/>Is a confidence function thereon, the fusion rule can be expressed by D-S fusion theory as:

。

wherein,called collision coefficient, is a normalization factor, +.>The larger the value, the stronger the collision, which is mathematically defined as follows:

。

representing impossible events +.>Is->And identifying frame elements corresponding to the classifiers.

Obtained according to the D-S fusion theoryFusion result of confidence functions of individual SVM classifier +.>Obtaining final classification result by maximum BPA (basic probability assignment) method, which is +.>：

。

The architecture of the whole unbalanced hyperspectral image integrated classification system is shown in fig. 2. In fig. 2, the original data set is divided into a training data set and a test data set after spectrum-space feature fusion, wherein the training data set needs to be re-balanced by re-sampling to obtain a re-balanced data set, that is, the obtained balanced sample set.

Based on the above description, the present invention has the following advantages over the prior art:

(1) The synthetic oversampling process is based on spectrum-space feature fusion, interpolation is carried out in a spectrum-space neighborhood, and space structure information in a hyperspectral image can be better utilized.

(2) According to the invention, the neighbor samples are randomly selected from the characteristic neighborhood and the space neighborhood of the seed samples to conduct interpolation to generate the synthesized samples, so that the diversity of the synthesized samples is improved, and the robustness and generalization performance of the classifier are improved.

(3) According to the invention, the projection matrix of the feature subset is obtained by using the Fisher discriminant analysis algorithm, and the optimization feature space capable of effectively relieving the problem of class overlap is calculated.

(4) The invention constructs the rotation matrixAnd alignment using feature vectorsThe method implements feature space transformation so that the data can be re-represented in the optimized feature space. The integrated classification is performed in a plurality of optimized feature spaces, so that better classification performance can be obtained.

(5) The invention fully considers the uncertainty problem caused by inconsistent output results of different classifiers, and utilizes the D-S theory to furthest alleviate the uncertainty, thereby realizing decision space optimization to obtain more reliable fusion results.

In addition, the self-adaptive artificial synthesis oversampling algorithm based on the spectrum-space neighborhood can be used for processing other special images with multi-channel characteristics such as multi-spectrum and the like, and the rebalancing of pixel samples in the images is realized. The class overlap feature optimization method provided by the invention can be popularized to other classification tasks with unbalanced data. In the class overlap feature optimization method and the rotation integration method based on class overlap feature optimization, other new algorithms capable of realizing class overlap optimization feature space searching can be used for replacing the FDA algorithm, and the rotation matrix construction method is modified correspondingly.

Still further, the present invention also provides an electronic device including: memory and a processor. The processor is connected with the memory. The memory is used for storing a computer program. The processor is used for retrieving and executing the computer program to implement the unbalanced hyperspectral image integrated classification method provided above.

Furthermore, the computer program in the above-described memory may be stored in a computer-readable storage medium when it is implemented in the form of a software functional unit and sold or used as a separate product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.

The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.

Claims (10)

1. An unbalanced hyperspectral image integrated classification method is characterized by comprising the following steps:

performing sample synthesis on a few samples in the original hyperspectral image data set according to a set sampling strategy in the fused spectrum-space characteristic space based on a spectrum-space neighborhood self-adaptive artificial synthesis oversampling algorithm to obtain a balanced sample set; the minority samples are samples with the class unbalance degree lower than a set value;

randomly dividing the characteristics in the balance sample set for a plurality of times to obtain a plurality of characteristic subsets;

solving a projection matrix of the feature subset obtained by random division through a Fisher discriminant analysis algorithm, and constructing a plurality of corresponding rotation matrices by using a reconstructed projection vector feature alignment method;

performing feature space transformation on the data matrix of the balanced sample set based on a plurality of rotation matrices to obtain a plurality of rotation sample sets;

and correspondingly inputting the rotation sample sets on each different feature space into a support vector machine base classifier, and integrating all support vector machine base classifiers by using D-S fusion to obtain classification results so as to finish decision space optimization.

2. The unbalanced hyperspectral image integrated classification method of claim 1, wherein the method is characterized in that a sample synthesis is performed on a few samples in an original hyperspectral image dataset according to a set sampling strategy in a fused spectrum-space feature space based on a spectrum-space neighborhood adaptive artificial synthesis oversampling algorithm, and a balanced sample set is obtained, and specifically comprises the following steps:

scanning a two-dimensional image in an original hyperspectral image dataset by using a sliding window to obtain a track matrix, and embedding space structure information into the track matrix to finish the fusion of spectrum-space characteristic space; the track matrix has a hanker-block-hanker structure, expressed as a hanker matrix with respect to the sub-matrix;

singular value decomposition is carried out on the basis of the track matrixes to obtain component matrixes, and the contribution of each component matrix to carrying track matrix information is determined;

grouping the component matrixes according to the contribution to obtain a grouping matrix;

recovering the Hank structure of each sub-matrix in the grouping matrix through the diagonal average operation of the auxiliary diagonal, recovering the Hank structure of each matrix formed by the sub-matrices recovering the Hank structure through the diagonal average operation of the auxiliary diagonal based on the sub-matrix, and completing two-step Hank processing to obtain a Hank-block-Hank matrix;

reconstructing two-dimensional signals based on the Hank-block-Hank matrix to obtain a reconstructed two-dimensional image matrix;

and carrying out sample resampling based on the reconstructed two-dimensional image matrix to obtain a balanced sample set.

3. The method for integrated classification of unbalanced hyperspectral images according to claim 2, wherein the step of obtaining a track matrix by scanning a two-dimensional image in an original hyperspectral image dataset by using a sliding window, and the step of embedding spatial structure information into the track matrix specifically comprises the following steps:

scanning a two-dimensional image in the original hyperspectral image dataset by using a sliding window to obtain a window matrix;

and vectorizing the window matrix and splicing the window matrix row by row to obtain the track matrix embedded with the space structure information.

4. The method for integrated classification of unbalanced hyperspectral images according to claim 2, wherein the method for integrated classification of unbalanced hyperspectral images is characterized in that the method comprises the steps of performing singular value decomposition based on the track matrixes to obtain component matrixes, and determining the contribution of each component matrix to carrying track matrix information, and specifically comprises the following steps:

obtaining a matrix product, and carrying out eigenvalue decomposition on the matrix product to obtain an eigenvalue and an eigenvector corresponding to the eigenvalue; the matrix product is the product of the trajectory matrix and a transpose matrix of the trajectory matrix;

determining a component matrix based on the eigenvalues and eigenvectors corresponding to the eigenvalues;

and taking the eigenvalue corresponding to the component matrix as the contribution.

5. The unbalanced hyperspectral image integrated classification method of claim 2, wherein grouping the component matrices by the contribution amounts results in grouping matrices, specifically comprising:

dividing the component matrix into a plurality of disjoint subsets;

and reserving a subset with the contribution quantity exceeding a set value in the plurality of disjoint subsets to obtain the grouping matrix.

6. The method for integrated classification of unbalanced hyperspectral images according to claim 2, wherein the two-dimensional signal reconstruction based on the hank-block-hank matrix is performed to obtain a reconstructed two-dimensional image matrix, and specifically comprises:

vectorizing each Hanker-block-Hanker matrix to obtain a vector;

and splicing the vectors in the row direction according to the auxiliary diagonal order to obtain a reconstructed two-dimensional image matrix.

7. The unbalanced hyperspectral image integrated classification method of claim 2, wherein the sample resampling is performed based on the reconstructed two-dimensional image matrix to obtain a balanced sample set, and specifically comprises:

determining a pixel sample vector based on the reconstructed two-dimensional image matrix;

determining the sample number of a certain class based on the class corresponding to the pixel sample vector;

determining a certain class of unbalance based on the number of samples;

when the unbalance is smaller than a balance threshold, setting an oversampling rate, and determining the number of samples to be synthesized in the category based on the oversampling rate;

determining a characteristic neighborhood of the sample by adopting a K nearest neighbor algorithm, and determining a space neighborhood according to the pixel space position to obtain a spectrum-space neighborhood of the sample;

determining the heterogeneous sample proportion of the sample in the spectrum-space neighborhood;

normalizing all samples in the category based on the heterogeneous sample proportion to obtain sampling probability that the samples in the category are selected as seed samples;

determining the number of synthesized samples using the samples belonging to the category as seed samples according to the sampling probability and the number of the samples to be synthesized;

and carrying out repeated random neighbor extraction and interpolation treatment on the seed samples according to the number of the synthesized samples to obtain synthesized samples of the seed samples so as to obtain the balanced sample set.

8. The method for integrated classification of unbalanced hyperspectral images according to claim 1, wherein the projection matrix of the feature subset is solved by a Fisher discriminant analysis algorithm, and a rotation matrix is constructed by using a modified projection vector feature alignment method, and specifically comprises the following steps:

constructing an all-zero matrix;

randomly extracting samples with set proportions from the characteristic subsets with places to obtain a training set;

solving a sample intra-class hash matrix and a sample inter-class hash matrix on a subspace for each feature subset;

determining all eigenvalues of a square matrix and eigenvectors corresponding to the eigenvalues based on the intra-sample-class hash matrix and the inter-sample-class hash matrix according to FDA criteria;

sorting the feature vectors according to the feature values to obtain a projection matrix;

determining a feature corpus index corresponding to the features in the projection matrix;

and placing the features in the projection matrix at the corresponding positions of the all-zero matrix according to the feature corpus index to obtain the rotation matrix.

9. An unbalanced hyperspectral image integrated classification system, characterized in that the system is used for implementing the unbalanced hyperspectral image integrated classification method according to any one of claims 1 to 8; the system comprises:

the sample space optimization module is used for carrying out sample synthesis on a few samples in the original hyperspectral image data set according to a set sampling strategy in the fused spectrum-space characteristic space based on a self-adaptive artificial synthesis oversampling algorithm of the spectrum-space neighborhood to obtain a balanced sample set; the minority samples are samples with the class unbalance degree lower than a set value;

the feature space optimization module is used for carrying out multiple random division on the features in the balance sample set to obtain a plurality of feature subsets, solving a projection matrix of the feature subsets obtained by the random division through a Fisher discriminant analysis algorithm, constructing a plurality of corresponding rotation matrixes by using a modified projection vector feature alignment method, and carrying out feature space transformation on a data matrix of the balance sample set based on the rotation matrixes to obtain a plurality of rotation sample sets;

and the decision space optimization module is used for correspondingly inputting the rotation sample sets on each different feature space into a support vector machine base classifier, and integrating all support vector machine base classifiers by using D-S fusion to obtain classification results so as to finish decision space optimization.

10. An electronic device, comprising:

a memory for storing a computer program;

a processor, coupled to the memory, for retrieving and executing the computer program to implement the unbalanced hyperspectral image integrated classification method of any one of claims 1-8.