CN113792809A - Remote sensing picture classification method based on random semi-supervised feature extraction model - Google Patents
Remote sensing picture classification method based on random semi-supervised feature extraction model Download PDFInfo
- Publication number
- CN113792809A CN113792809A CN202111101521.2A CN202111101521A CN113792809A CN 113792809 A CN113792809 A CN 113792809A CN 202111101521 A CN202111101521 A CN 202111101521A CN 113792809 A CN113792809 A CN 113792809A
- Authority
- CN
- China
- Prior art keywords
- network
- feature extraction
- semi
- layer
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 99
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000012549 training Methods 0.000 claims abstract description 64
- 238000005457 optimization Methods 0.000 claims abstract description 13
- 238000009826 distribution Methods 0.000 claims description 29
- 230000006870 function Effects 0.000 claims description 12
- 238000012706 support-vector machine Methods 0.000 claims description 10
- 238000012952 Resampling Methods 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 9
- 238000005070 sampling Methods 0.000 claims description 9
- 238000013145 classification model Methods 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 238000009827 uniform distribution Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 2
- 238000013527 convolutional neural network Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 1
- 235000006173 Larrea tridentata Nutrition 0.000 description 1
- 244000073231 Larrea tridentata Species 0.000 description 1
- 230000003042 antagnostic effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 229910003460 diamond Inorganic materials 0.000 description 1
- 239000010432 diamond Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a remote sensing picture classification method based on a random semi-supervised feature extraction model, which comprises the following steps: a. establishing a remote sensing scene picture database; b. constructing a random semi-supervised feature extraction model; c. constructing a classification network; d. establishing an optimization target of a random semi-supervised feature extraction model; e. game alternative training random semi-supervised feature extraction model; f. training a classifier; g. and finishing the task of classifying the remote sensing pictures. The method can effectively improve the utilization rate of mass weakly labeled remote sensing data and improve the task performance of remote sensing image classification.
Description
Technical Field
The invention relates to a remote sensing picture classification method based on a random semi-supervised feature extraction model.
Background
With the improvement of the spatial resolution of the remote sensing data, the scene classification task becomes a research hotspot of the remote sensing image classification task. The remote sensing image scene classification is to correctly label a given scene image with predefined semantic categories. In the prior art, a method based on deep learning is generally adopted for image classification. The method is widely applied to the picture classification task of life scenes, and reaches or surpasses the level of human beings. However, the remote sensing scene classification algorithm based on deep learning mainly focuses on supervised learning, which requires a large amount of labeled data. However, compared with the life scene pictures, the remote sensing pictures have large intra-class difference, small inter-class separability and multi-scale targets, and the remote sensing data samples have high labeling cost and strong specialization and show the characteristics of massive weak labels, so that the remote sensing picture classification task faces a larger challenge.
The existing methods based on deep learning are mainly divided into three categories: convolutional neural network CNN based methods, variational self-encoders VAE based methods and methods based on generation of antagonistic networks GAN. In the face of a large number of remote sensing images, CNN-based methods require the use of a large number of labeled samples to train the model or fine tune a pre-trained convolutional neural network. However, the remote sensing data sample labeling cost is high, and the specialization is strong, so that the application is not ideal. Although the method based on the VAE is an unsupervised generation type model learning method, the method has good effect in the scene classification of the remote sensing image, but the general feature representation of the original image cannot be learned sometimes in the process of reconstructing the similar image through the input image, so most of the VAE-based methods cannot learn the optimal distinguishable features of different scene classes. GAN-based methods are another generative model learning method that can be used unsupervised or semi-supervised, which is widely used in the task of remote sensing images.
For example, patent CN111339935A discloses an optical remote sensing image classification method based on an interpretable CNN image classification model, which focuses mainly on the interpretability problem of a deep learning model, and improves the accuracy of remote sensing image classification by proposing an interpretable CNN image classification model. For another example, patent CN108596248 discloses a remote sensing image classification model based on an improved deep convolutional neural network, which provides an improved deep convolutional neural network, and through dimension reduction, convolutional multi-channel optimization, feature extraction capability improvement and wave-band processing of spatial location features on a remote sensing feature image, the computing resource consumption of the deep convolutional neural network is reduced, meanwhile, the feature extraction effect is ensured, and the recognition degree of the spatial location features is improved. Therefore, the technologies are proposed around the remote sensing classification problem, the purpose is to reduce the consumption of computing resources and improve the remote sensing classification effect, and all the adopted technologies are fully supervised models, so that a large amount of labeled data is needed.
Of course, there are also some techniques proposed to classify pictures using a generation countermeasure network. The generation countermeasure network is composed of a generation network and a discrimination network, and deep feature representation of the data is learned under the condition of no-marking or weak-marking training data through a countermeasure training process between the generation network and the discrimination network. In the prior art, the generation of the countermeasure network is widely applied to the typical picture classification task of the life scene, and a competitive result is obtained. However, the mode of generating the countermeasure network is not widely applied to the remote sensing scene classification task, and the remote sensing picture has the characteristics of large intra-class difference, small inter-class separability, multi-scale target and the like compared with a typical life scene picture, so that the remote sensing picture classification task still faces a larger challenge.
Disclosure of Invention
The invention aims to provide a remote sensing picture classification method based on a random semi-supervised feature extraction model.
In order to achieve the purpose, the invention provides a remote sensing picture classification method based on a random semi-supervised feature extraction model, which comprises the following steps:
a. establishing a remote sensing scene picture database;
b. constructing a random semi-supervised feature extraction model;
c. constructing a classification network;
d. establishing an optimization target of a random semi-supervised feature extraction model;
e. game alternative training random semi-supervised feature extraction model;
f. training a classifier;
g. and finishing the task of classifying the remote sensing pictures.
According to one aspect of the invention, in the step (a), remote sensing scene picture data is collected and labeled according to land use types, then all data are divided into labeled data sets and unlabeled data sets according to whether category labels are provided, wherein one part of the labeled data sets is used as test data and does not participate in training, the rest part and all unlabeled data form a training data set, and finally data enhancement is performed on the training data set in a mode of horizontally overturning, vertically overturning and rotating by 90 degrees.
According to an aspect of the present invention, the constructing of the random semi-supervised feature extraction model in the step (b) includes constructing a random generation network G and a semi-supervised feature extraction network D;
the random generation network G comprises an input noise layer, a random layer, a resampling layer, a deconvolution layer and an output picture layer;
the semi-supervised feature extraction network D comprises an input picture layer, a convolution layer, a feature layer, a full connection layer and an output layer;
the optimization goal of the randomly generated network G is to find α so that:
wherein p iszThe method comprises the following steps that pre-defined distribution obeyed by an input variable z, epsilon is a middle variable of a random layer, and obey Gaussian distribution N (0, I), an output variable of the random layer obeys Gaussian prior distribution, alpha is a parameter to be trained, f (·) represents an output feature of a picture after semi-supervised feature extraction network D, E is a mathematical expectation about the variable, and x is an input picture;
the semi-supervised feature extraction network D is optimized to find η so that:
wherein eta is a parameter to be trained of the feature extraction network; y isiAn ith dimension component representing label y; di() represents the ith dimension component of the network output, K is the total number of classes of the classification task;
the random generation network G is constructed by the following steps:
b11, randomly sampling noise from a predefined distribution as an input of the random generation network G;
b12, setting the widths and depths of the random layer, the resampling layer and the deconvolution layer according to the specific task difficulty, wherein the output variables of the random layer obey Gaussian prior distribution;
b13, outputting pseudo data with the same size as the original data and serving as the input of the semi-supervised feature extraction network D;
when constructing the semi-supervised feature extraction network D:
in the game training stage, inputting pictures including real labeled data, real unlabelled data and generated pseudo data, and outputting values including true and false logic output and picture category output;
in the classifier training stage, the input of the semi-supervised feature extraction network D is labeled data, and the output is corresponding high-dimensional features which are used as the input of a classifier;
in the stage of realizing the remote sensing classification task, the input of the semi-supervised feature extraction network D is a remote sensing picture to be classified, and the output is corresponding high-dimensional features;
the random generation network G comprises 9 layers in total, and the first layer is an input layer; the second layer is a random layer and consists of two identical fully-connected networks and is used for learning the mean value and the variance of Gaussian prior distribution obeyed by the output variable of the layer; the third layer is a resampling layer; the subsequent five layers are deconvolution layers, wherein the size of a convolution kernel is 4 multiplied by 4, and the step length is 2; the last layer is an output layer, and 3-channel pictures with the size of 256 multiplied by 256 are output; the deconvolution layers all adopt ReLU-form activation functions, and the output layer adopts tanh-type activation functions;
the semi-supervised feature extraction network D comprises 9 layers, wherein the first layer is a picture input layer, and input pictures comprise marked real pictures, unmarked real pictures and generated pictures; the subsequent six layers are convolution layers, wherein the size of a convolution kernel is 5 multiplied by 5, the step length is 2, the activation function is LEAKYRELU, and the parameter is 0.2; the characteristic layer combines the characteristic information of the previous three layers; the last layer is a full connection layer, and the category or the authenticity information of the picture is output.
According to an aspect of the present invention, in the step (C), a linear support vector machine network is used to construct the classification network, and the regularization parameter C is 1000.
According to an aspect of the present invention, in step (D), the optimization goal of establishing the random semi-supervised feature extraction model is game-play-a maximum and minimum value about the randomly generated network G and the semi-supervised feature extraction network D, and is composed of supervised learning loss and unsupervised learning loss, as follows:
wherein p is the true remote sensing data distribution; p is a radical ofGIs the distribution of the generated data of the randomly generated network G; p is a radical ofD(y | x, y ≦ K) represents the probability that the input picture x is determined to be the label y; p is a radical ofD(y ≦ K | x) represents the probability that the input picture x is real data; p is a radical ofD(K +1| x) represents the probability that the input picture x is generating dummy data, J is a loss function with respect to networks G and D.
According to one aspect of the present invention, in said step (e), the training goal of the semi-supervised feature extraction network D is to maximize the probability that the labeled data classification is correct, by minimizing the cross-entropy
To realize the operation; maximizing the probability that the true data is judged true and the generated data is judged false by minimizing
And
to realize the operation;
the training goal of the randomly generated network G is to minimize the probability that the generated data is judged to be false data by minimizing
To realize the operation; while minimizing the distance between the generated data features and the real data features by minimizing
To realize the operation;
and repeating the alternate training between the semi-supervised feature extraction network D and the random generation network G until reaching the specified training step number, and storing the trained random semi-supervised feature extraction network.
According to one aspect of the invention, the step (e) is game alternate training according to the following steps:
e1, setting the total training round number N to be 200, randomly generating a network G to update k to be 2 times every time the semi-supervised feature extraction network D updates, and optimizing the learning rate lambda and the momentum parameter beta in the parameter of the Adam optimization algorithm1λ ═ 0.0002 and β, respectively10.5, small batch size m 64, hidden spatial dimension d 100, and for UC-merceded dataset K21;
e2, randomly initializing network parameters eta and alpha;
e3, training N times alternately according to the following modes:
e 31: randomly sampling m hidden variables z1，...，zm}～Ud[-1，1]And the Gaussian variable { ∈1，...，∈mN (0, I), calculating and judging the loss L of the network about the generated pictureD1：
e 32: randomly selecting m true samples { x ] from the label-free training dataset1，...，xmCalculating the loss L of the discrimination network about the unmarked pictureD2：
e 33: randomly selecting m true samples from a labeled training dataset (x)1，y1)，...，(xm，ym) Calculating the loss L of the marked picture of the judgment networkD3：
e 34: updating a semi-supervised feature extraction network parameter eta:
e 35: randomly sampling m hidden variables z1，...，zm}～Ud[-1，1]And the Gaussian variable { ∈1，...，∈mN (0, I), calculating the true and false loss L of the random generation network G about the generated pictureG1：
e 36: randomly selecting m true samples { x ] from the label-free training dataset1，...，xmCalculating the characteristic loss L of the random generation network G about the generated pictureG2：
e 37: updating the randomly generated network parameter α:
e 38: repeating said steps (e35) to (e37) k times, returning to said step (e 31);
e4, storing the trained random semi-supervised feature extraction network;
wherein U is uniformly distributed, I is a unit matrix +ηIs a gradient on a network parameter η +αIs a gradient with respect to the network parameter alpha.
According to one aspect of the present invention, in step (f), all the labeled data in the training data set are input into the trained random semi-supervised feature extraction network to obtain corresponding high-dimensional features, and the features and the label information are input into the support vector machine network for training.
According to one aspect of the invention, in the step (g), the trained random semi-supervised feature extraction model and the support vector machine network are saved to form a classification model;
selecting remote sensing data to be classified from the test data set, and preprocessing the remote sensing data to be classified;
inputting the processed picture into a trained random semi-supervised feature extraction network to obtain corresponding high-dimensional features;
and inputting the high-dimensional features into a trained classifier, outputting the class information of the remote sensing picture, and completing the task of classifying the remote sensing picture.
According to one aspect of the invention, the predefined distribution of input variables is a uniform distribution between [ -1,1 ].
According to the concept of the invention, the remote sensing picture classification method based on the random semi-supervised feature extraction model is provided aiming at the characteristics of complex data distribution and mass weak labeling of the remote sensing picture.
According to the scheme of the invention, the random semi-supervised feature extraction network with higher high-dimensional feature capability for capturing the remote sensing data is obtained through game alternate training of the random generation network with higher expression capability and the semi-supervised feature extraction network, so that the utilization rate of mass weakly labeled remote sensing data is effectively improved, and the game alternate training is finally used for improving the task performance of remote sensing image classification.
Drawings
FIG. 1 is a schematic flow chart of a remote sensing picture classification method based on a random semi-supervised feature extraction model according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a randomly generated network architecture in a remote sensing picture classification method based on a random semi-supervised feature extraction model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a semi-supervised feature extraction network architecture in a remote sensing picture classification method based on a random semi-supervised feature extraction model according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a confusion matrix result of a surface feature scene classification task in a remote sensing image classification method based on a random semi-supervised feature extraction model according to an embodiment of the present invention.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
The present invention is described in detail below with reference to the drawings and the specific embodiments, which are not repeated herein, but the embodiments of the present invention are not limited to the following embodiments.
Referring to fig. 1, the invention discloses a remote sensing picture classification method based on a random semi-supervised feature extraction model, relates to the technical field of remote sensing image interpretation, and belongs to the key intelligent theory and algorithm research of a space intelligent system. The method comprises the steps of firstly establishing a remote sensing scene picture database, establishing a random semi-supervised feature extraction model, establishing a classification network, establishing an optimization target of the random semi-supervised feature extraction model, training the random semi-supervised feature extraction model in a game alternating mode, training a classifier and finishing a remote sensing picture classification task.
In the embodiment, in order to show the effect and the capability of the method for processing the scene classification task of the remote sensing picture, a UC-merceded remote sensing data set is taken as an example to complete the scene classification task of 21 types of ground objects. UC-Merced data set, comprising 21 land use categories, as shown in fig. 4, 1: mobile home park; 2: beach (beach); 3: tenis court; 4: airplane (airplane); 5: dense residential area; 6: harbor (seaport); 7: building; 8: forest; 9: intersectional (crossroad); 10: river (river); 11: sparse residence; 12: runway (runway); 13: park lot; 14: baseball Diamond (baseball field); 15: agricultural (agricultural); 16: storage tanks; 17: chaparral (jungle); 18: golf course; 19: freeway (expressway); 20: medium residential area; 21: overpass (overpass). Each class consists of 100 aerial images, 256 × 256 pixels in size. Of course, the method of the embodiment is mainly used for completing the task of classifying the remote sensing picture, and in other embodiments, other interpretation tasks based on picture features, such as a target detection task, can also be completed by using the method of the present invention.
In the invention, when the remote sensing scene picture database is established, the remote sensing scene picture data is collected and is labeled according to the land use type, and then the remote sensing scene picture data is divided into a labeled data set and a non-labeled data set according to whether a category label is provided. Of course, data division is also required, and for the data set of this embodiment, specifically, all samples in the data set are randomly divided into training data (accounting for 80%, 1680) and test data (accounting for 20%, 420). In addition, because the data set comprises more categories, and each category only comprises 100 pictures, the training data set also needs to be subjected to data enhancement, specifically, 3 times of enhanced data is obtained by means of horizontal turning, vertical turning and 90-degree rotation, the extended training data set comprises 6720 pictures in total by combining the original data, and because the data set is labeled data, the invention adopts a form of removing corresponding labels of the training data set to construct a required label-free data set.
In the invention, the random semi-supervised feature extraction model construction comprises a random generation network G and a semi-supervised feature extraction network D.
As shown in fig. 2, the random generation network G includes an input noise layer, a random layer, a resampling layer, a deconvolution layer, and an output picture layer. The random generation network G comprises 9 layers in total, and the first layer is an input layer; the second layer is a random layer and consists of two identical fully-connected networks and is used for learning the mean value and the variance of Gaussian prior distribution obeyed by the output variable of the layer; the third layer is a resampling layer; the subsequent five layers are deconvolution layers, wherein the size of a convolution kernel is 4 multiplied by 4, and the step length is 2; the last layer is an output layer, and 3-channel pictures with the size of 256 multiplied by 256 are output; the deconvolution layers all adopt ReLU-form activation functions, and the output layer adopts tanh-type activation functions;
the optimization goal of the randomly generated network G is to find α so that:
wherein p iszThe method comprises the steps of obtaining a pre-defined distribution obeyed by an input variable z, representing an intermediate variable of a random layer by epsilon, obeying a Gaussian distribution N (0, I), obtaining an output variable of the random layer by obeying a Gaussian prior distribution, wherein alpha is a parameter to be trained, f (-) represents an output characteristic of a picture after semi-supervised characteristic extraction network D, E is a mathematical expectation about the variable, and x is an input picture.
When the random generation network G is constructed, random sampling noise in predefined distribution is used as input of the random generation network G, the widths and depths of a random layer, a resampling layer and a deconvolution layer are set according to specific task difficulty, output variables of the random layer obey Gaussian prior distribution, and finally pseudo data with the same size as original data are output and used as input of a semi-supervised feature extraction network D.
As shown in fig. 3, the semi-supervised feature extraction network D includes an input picture layer, a convolutional layer, a feature layer, a full connection layer and an output layer. The semi-supervised feature extraction network D comprises 9 layers, wherein the first layer is a picture input layer, and input pictures comprise marked real pictures, unmarked real pictures and generated pictures; the subsequent six layers are convolution layers, wherein the size of a convolution kernel is 5 multiplied by 5, the step length is 2, the activation function is LEAKYRELU, and the parameter is 0.2; the characteristic layer combines the characteristic information of the previous three layers; the last layer is a full connection layer, and the category or the authenticity information of the picture is output.
The semi-supervised feature extraction network D is optimized to find η so that:
wherein eta is a parameter to be trained of the feature extraction network; y isiAn ith dimension component representing label y; di(. cndot.) represents the ith dimension component of the network output, and K is the total number of classes for the classification task.
When a semi-supervised feature extraction network D is constructed, in a game training stage, inputting pictures including real labeled data, real unlabelled data and generated pseudo data, and outputting values including true and false logic output and picture category output; in the classifier training stage, the input of the semi-supervised feature extraction network D is labeled data, and the output is corresponding high-dimensional features which are used as the input of a classifier; in the stage of realizing the remote sensing classification task, the input of the semi-supervised feature extraction network D is a remote sensing picture to be classified, and the output is corresponding high-dimensional features.
In the invention, a classification network is constructed by adopting a linear support vector machine network, and the regularization parameter C is 1000.
In the invention, the optimization target for establishing the random semi-supervised feature extraction model is the problem of the maximum and minimum values of a game about a randomly generated network G and a semi-supervised feature extraction network D, and the random semi-supervised feature extraction model consists of two parts of supervised learning loss and unsupervised learning loss, and specifically comprises the following steps:
wherein p is the true remote sensing data distribution; p is a radical ofGIs the distribution of the generated data of the randomly generated network G; p is a radical ofD(y | x, y ≦ K) represents the probability that the input picture x is determined to be the label y; p is a radical ofD(y ≦ K | x) represents the probability that the input picture x is real data; p is a radical ofD(K +1| x) represents the probability that the input picture x is generating dummy data, J is a loss function with respect to networks G and D.
In the invention, the training goal of the semi-supervised feature extraction network D is to maximize the probability of correct classification of labeled data by minimizing the cross entropy
To realize the operation; simultaneously maximizing the probability that the true data is judged to be true and the generated data is judged to be false, by minimizing
And
to be implemented.
The training goal of the randomly generated network G is to minimize the probability that the generated data is judged to be false data by minimizing
To realize the operation; while minimizing the distance between the generated data features and the real data features by minimizing
To be implemented.
And repeating the alternate training between the semi-supervised feature extraction network D and the random generation network G until reaching the specified training step number, and storing the trained random semi-supervised feature extraction network.
When the game alternative training is carried out, firstly, parameter setting is carried out, specifically, the total training round number N is set to be 200, the semi-supervised feature extraction network D randomly generates the network G to update k to be 2 times every time the semi-supervised feature extraction network D updates, and Adam optimizes algorithm parameters (the learning rate lambda is 0.0002, and the momentum parameter beta is 0.0002)10.5), mini-batch size m 64, hidden spatial dimension d 100, and K for UC-merceded dataset21. Network parameters η and α are randomly initialized. Alternately training in the following way:
for total number of training rounds, ndo
Randomly sampling m hidden variables z1，...，zm}～Ud[-1，1]And the Gaussian variable { ∈1，...，∈mN (0, I), calculating and judging the loss L of the network about the generated pictureD1：
Randomly selecting m true samples { x ] from the label-free training dataset1，...，xmCalculating the loss L of the discrimination network about the unmarked pictureD2：
Randomly selecting m true samples from a labeled training dataset (x)1，y1)，...，(xm，ym) Calculating the loss L of the marked picture of the judgment networkD3：
Updating a semi-supervised feature extraction network parameter eta:
randomly sampling m hidden variables z1，...，zm}～Ud[-1，1]And the Gaussian variable { ∈1，...，∈mN (0, I), calculating the true and false loss L of the random generation network G about the generated pictureG1：
Randomly selecting m true samples { x ] from the label-free training dataset1，...，xmCalculating the characteristic loss L of the random generation network G about the generated pictureG2：
Updating the randomly generated network parameter α:
repeating the steps to calculate the loss LG1And characteristic loss LG2And updating the randomly generated network parameter alpha for a plurality of times (k), namely, the picture loss L can be calculatedD1A step (2);
end for
and finally, storing the trained random semi-supervised feature extraction network.
Wherein U is uniformly distributed, I is a unit matrix,
in order to be a gradient with respect to the network parameter η,
is a gradient with respect to the network parameter alpha.
In the invention, when training a classifier, all labeled data of a training data set are input into a trained random semi-supervised feature extraction network to obtain corresponding high-dimensional features, and the features and label information are input into a support vector machine network to train the support vector machine network.
In the invention, a trained random semi-supervised feature extraction model and a support vector machine network are stored to form a classification model, the remote sensing picture to be classified is input into the classification model, and finally the class information of the remote sensing picture is output. Specifically, firstly, remote sensing data to be classified is selected from the test data set, the remote sensing data to be classified is preprocessed, and the pictures are uniformly cut into 256 × 256 pictures. And then inputting the processed picture into a trained random semi-supervised feature extraction network to obtain corresponding high-dimensional features. And finally, inputting the high-dimensional features into a trained classifier, and finally outputting the class information of the remote sensing picture, thereby completing the task of classifying the remote sensing picture.
In the present invention, the predefined distribution of input variables is a uniform distribution between [ -1,1 ].
The average classification accuracy of the method can reach more than 98%, and a confusion matrix of the classification result is shown in figure 4.
In summary, the invention combines the randomly generated countermeasure network with stronger expression capability and the semi-supervision method by utilizing the capability of generating the distribution of the countermeasure network capturing data and the advantage of processing mass weak annotation data by the semi-supervision method, is applied to the remote sensing picture classification task, can effectively utilize mass weak annotation remote sensing data, has stronger picture characteristic extraction capability, can be used for completing the remote sensing picture classification task, and can also be widely applied to other remote sensing interpretation tasks based on picture characteristics, such as target detection, ground feature segmentation and the like.
The above description is only one embodiment of the present invention, and is not intended to limit the present invention, and it is apparent to those skilled in the art that various modifications and variations can be made in the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A remote sensing picture classification method based on a random semi-supervised feature extraction model comprises the following steps:
a. establishing a remote sensing scene picture database;
b. constructing a random semi-supervised feature extraction model;
c. constructing a classification network;
d. establishing an optimization target of a random semi-supervised feature extraction model;
e. game alternative training random semi-supervised feature extraction model;
f. training a classifier;
g. and finishing the task of classifying the remote sensing pictures.
2. The method according to claim 1, wherein in the step (a), the remote sensing scene picture data is collected and labeled according to land use types, all data are divided into labeled data sets and unlabeled data sets according to whether category labels are provided, wherein one part of the labeled data sets is used as test data and is not involved in training, the rest part and all unlabeled data form a training data set, and finally the training data set is subjected to data enhancement in a mode of horizontal overturning, vertical overturning and 90-degree rotation.
3. The method of claim 1, wherein the constructing a random semi-supervised feature extraction model in step (b) comprises constructing a random generation network G and a semi-supervised feature extraction network D;
the random generation network G comprises an input noise layer, a random layer, a resampling layer, a deconvolution layer and an output picture layer;
the semi-supervised feature extraction network D comprises an input picture layer, a convolution layer, a feature layer, a full connection layer and an output layer;
the optimization goal of the randomly generated network G is to find α so that:
wherein p iszThe method comprises the following steps that pre-defined distribution obeyed by an input variable z, epsilon is a middle variable of a random layer, and obey Gaussian distribution N (0, I), an output variable of the random layer obeys Gaussian prior distribution, alpha is a parameter to be trained, f (·) represents an output feature of a picture after semi-supervised feature extraction network D, E is a mathematical expectation about the variable, and x is an input picture;
the semi-supervised feature extraction network D is optimized to find η so that:
wherein eta is a parameter to be trained of the feature extraction network; y isiAn ith dimension component representing label y; di() represents the ith dimension component of the network output, K is the total number of classes of the classification task;
the random generation network G is constructed by the following steps:
b11, randomly sampling noise from a predefined distribution as an input of the random generation network G;
b12, setting the widths and depths of the random layer, the resampling layer and the deconvolution layer according to the specific task difficulty, wherein the output variables of the random layer obey Gaussian prior distribution;
b13, outputting pseudo data with the same size as the original data and serving as the input of the semi-supervised feature extraction network D;
when constructing the semi-supervised feature extraction network D:
in the game training stage, inputting pictures including real labeled data, real unlabelled data and generated pseudo data, and outputting values including true and false logic output and picture category output;
in the classifier training stage, the input of the semi-supervised feature extraction network D is labeled data, and the output is corresponding high-dimensional features which are used as the input of a classifier;
in the stage of realizing the remote sensing classification task, the input of the semi-supervised feature extraction network D is a remote sensing picture to be classified, and the output is corresponding high-dimensional features;
the random generation network G comprises 9 layers in total, and the first layer is an input layer; the second layer is a random layer and consists of two identical fully-connected networks and is used for learning the mean value and the variance of Gaussian prior distribution obeyed by the output variable of the layer; the third layer is a resampling layer; the subsequent five layers are deconvolution layers, wherein the size of a convolution kernel is 4 multiplied by 4, and the step length is 2; the last layer is an output layer, and 3-channel pictures with the size of 256 multiplied by 256 are output; the deconvolution layers all adopt ReLU-form activation functions, and the output layer adopts tanh-type activation functions;
the semi-supervised feature extraction network D comprises 9 layers, wherein the first layer is a picture input layer, and input pictures comprise marked real pictures, unmarked real pictures and generated pictures; the subsequent six layers are convolution layers, wherein the size of a convolution kernel is 5 multiplied by 5, the step length is 2, the activation function is LEAKYRELU, and the parameter is 0.2; the characteristic layer combines the characteristic information of the previous three layers; the last layer is a full connection layer, and the category or the authenticity information of the picture is output.
4. The method of claim 1, wherein in step (C), a linear support vector machine network is used to construct the classification network, and the regularization parameter C is 1000.
5. The method of claim 1, wherein in step (D), the optimization goal of establishing the random semi-supervised feature extraction model is to game a maximum and minimum value about the randomly generated network G and the semi-supervised feature extraction network D, and is composed of supervised learning loss and unsupervised learning loss, as follows:
wherein p is the true remote sensing data distribution; p is a radical ofGIs the distribution of the generated data of the randomly generated network G; p is a radical ofD(y | x, y ≦ K) represents the probability that the input picture x is determined to be the label y; p is a radical ofD(y ≦ K | x) represents the probability that the input picture x is real data; p is a radical ofD(K +1| x) represents the probability that the input picture x is generating dummy data, J is a loss function with respect to networks G and D.
6. The method of claim 1, wherein in step (e), the training goal of the semi-supervised feature extraction network D is to maximize the probability that the labeled data classes are correct by minimizing cross entropy
To realize the operation; maximizing the probability that the true data is judged true and the generated data is judged false by minimizing
To know
To realize the operation;
the training goal of the randomly generated network G is to minimize the probability that the generated data is judged to be false data by minimizing
To realize the operation; while minimizing the distance between the generated data features and the real data features by minimizing
To realize the operation;
and repeating the alternate training between the semi-supervised feature extraction network D and the random generation network G until reaching the specified training step number, and storing the trained random semi-supervised feature extraction network.
7. The method of claim 6, wherein the step (e) comprises the steps of:
e1, setting the total training round number N to be 200, randomly generating a network G to update k to be 2 times every time the semi-supervised feature extraction network D updates, and optimizing the learning rate lambda and the momentum parameter beta in the parameter of the Adam optimization algorithm1λ ═ 0.0002 and β, respectively10.5, small batch size m 64, hidden spatial dimension d 100, and for UC-merceded dataset K21;
e2, randomly initializing network parameters eta and a;
e3, training N times alternately according to the following modes:
e 31: randomly sampling m hidden variables z1，...，zm}～Ud[-1，1]And the Gaussian variable { ∈1，...，∈mN (0, I), calculating and judging the loss L of the network about the generated pictureD1：
e 32: randomly selecting m true samples { x ] from the label-free training dataset1，...，xmCalculating the loss L of the discrimination network about the unmarked pictureD2：
e 33: randomly selecting m true samples from a labeled training dataset (x)1，y1)，...，(xm，ym) Calculating the loss L of the marked picture of the judgment networkD3：
e 34: updating a semi-supervised feature extraction network parameter eta:
e 35: randomly sampling m hidden variables z1，...，zm}～Ud[-1，1]And the Gaussian variable { ∈1，...，∈mN (0, I), calculating the true and false loss L of the random generation network G about the generated pictureG1：
e 36: randomization from unlabeled training data setsSelect m true samples { x1，...，xmCalculating the characteristic loss L of the random generation network G about the generated pictureG2：
e 37: updating the randomly generated network parameter α:
e 38: repeating said steps (e35) to (e37) k times, returning to said step (e 31);
e4, storing the trained random semi-supervised feature extraction network;
wherein U is uniformly distributed, I is a unit matrix,
in order to be a gradient with respect to the network parameter η,
is a gradient with respect to the network parameter alpha.
8. The method according to claim 1, wherein in step (f), all labeled data in the training data set are input into a trained random semi-supervised feature extraction network to obtain corresponding high-dimensional features, and the features and label information are input into a support vector machine network for training.
9. The method of claim 1, wherein in step (g), the trained stochastic semi-supervised feature extraction model and the support vector machine network are saved to form a classification model;
selecting remote sensing data to be classified from the test data set, and preprocessing the remote sensing data to be classified;
inputting the processed picture into a trained random semi-supervised feature extraction network to obtain corresponding high-dimensional features;
and inputting the high-dimensional features into a trained classifier, outputting the class information of the remote sensing picture, and completing the task of classifying the remote sensing picture.
10. The method according to any of claims 1-9, wherein the predefined distribution of input variables is a uniform distribution between [ -1,1 ].
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111101521.2A CN113792809B (en) | 2021-09-18 | 2021-09-18 | Remote sensing picture classification method based on random semi-supervised feature extraction model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111101521.2A CN113792809B (en) | 2021-09-18 | 2021-09-18 | Remote sensing picture classification method based on random semi-supervised feature extraction model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113792809A true CN113792809A (en) | 2021-12-14 |
| CN113792809B CN113792809B (en) | 2024-07-02 |
Family
ID=79184168
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111101521.2A Active CN113792809B (en) | 2021-09-18 | 2021-09-18 | Remote sensing picture classification method based on random semi-supervised feature extraction model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113792809B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117409262A (en) * | 2023-12-14 | 2024-01-16 | 厦门瑞为信息技术有限公司 | Method for quickly constructing image classification model based on CLIP |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102129571A (en) * | 2011-01-31 | 2011-07-20 | 重庆大学 | Method for classifying multi-spectral remote sensing data land use based on semi-supervisor manifold learning |
| CN102938073A (en) * | 2012-10-23 | 2013-02-20 | 中国人民解放军空军工程大学 | Method for classifying remote sensing images |
| US20130346346A1 (en) * | 2012-06-21 | 2013-12-26 | Microsoft Corporation | Semi-supervised random decision forests for machine learning |
| CN107358142A (en) * | 2017-05-15 | 2017-11-17 | 西安电子科技大学 | Polarimetric SAR Image semisupervised classification method based on random forest composition |
| CN108038448A (en) * | 2017-12-13 | 2018-05-15 | 河南理工大学 | Semi-supervised random forest Hyperspectral Remote Sensing Imagery Classification method based on weighted entropy |
| US20200019817A1 (en) * | 2018-07-11 | 2020-01-16 | Harbin Institute Of Technology | Superpixel classification method based on semi-supervised k-svd and multiscale sparse representation |
-
2021
- 2021-09-18 CN CN202111101521.2A patent/CN113792809B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102129571A (en) * | 2011-01-31 | 2011-07-20 | 重庆大学 | Method for classifying multi-spectral remote sensing data land use based on semi-supervisor manifold learning |
| US20130346346A1 (en) * | 2012-06-21 | 2013-12-26 | Microsoft Corporation | Semi-supervised random decision forests for machine learning |
| CN102938073A (en) * | 2012-10-23 | 2013-02-20 | 中国人民解放军空军工程大学 | Method for classifying remote sensing images |
| CN107358142A (en) * | 2017-05-15 | 2017-11-17 | 西安电子科技大学 | Polarimetric SAR Image semisupervised classification method based on random forest composition |
| CN108038448A (en) * | 2017-12-13 | 2018-05-15 | 河南理工大学 | Semi-supervised random forest Hyperspectral Remote Sensing Imagery Classification method based on weighted entropy |
| US20200019817A1 (en) * | 2018-07-11 | 2020-01-16 | Harbin Institute Of Technology | Superpixel classification method based on semi-supervised k-svd and multiscale sparse representation |
Non-Patent Citations (1)
| Title |
|---|
| 吴俊峰;姜志国;张浩鹏;蔡博文;罗鹏浩: "半监督条件随机场的高光谱遥感图像分类", 遥感学报, no. 004, 31 December 2017 (2017-12-31) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117409262A (en) * | 2023-12-14 | 2024-01-16 | 厦门瑞为信息技术有限公司 | Method for quickly constructing image classification model based on CLIP |
| CN117409262B (en) * | 2023-12-14 | 2024-03-01 | 厦门瑞为信息技术有限公司 | Method for quickly constructing image classification model based on CLIP |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113792809B (en) | 2024-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Building-a-nets: Robust building extraction from high-resolution remote sensing images with adversarial networks | |
| Wang et al. | RSNet: The search for remote sensing deep neural networks in recognition tasks | |
| US11195051B2 (en) | Method for person re-identification based on deep model with multi-loss fusion training strategy | |
| CN113221641B (en) | Video pedestrian re-identification method based on generation of antagonism network and attention mechanism | |
| CN107169504B (en) | A kind of hand-written character recognition method based on extension Non-linear Kernel residual error network | |
| CN107092661A (en) | A kind of image search method based on depth convolutional neural networks | |
| CN111611972B (en) | Crop leaf type identification method based on multi-view multi-task integrated learning | |
| CN109002755B (en) | Age estimation model construction method and estimation method based on face image | |
| CN111695640B (en) | Foundation cloud picture identification model training method and foundation cloud picture identification method | |
| Banimelhem et al. | Multi-thresholding image segmentation using genetic algorithm | |
| Badawi et al. | A hybrid memetic algorithm (genetic algorithm and great deluge local search) with back-propagation classifier for fish recognition | |
| Vallet et al. | A multi-label convolutional neural network for automatic image annotation | |
| CN117079098A (en) | Space small target detection method based on position coding | |
| CN112766062A (en) | Human behavior identification method based on double-current deep neural network | |
| CN112883931A (en) | Real-time true and false motion judgment method based on long and short term memory network | |
| Bounsaythip et al. | Genetic algorithms in image processing-a review | |
| CN107529647B (en) | Cloud picture cloud amount calculation method based on multilayer unsupervised sparse learning network | |
| CN114882278A (en) | Tire pattern classification method and device based on attention mechanism and transfer learning | |
| CN114612450A (en) | Image detection segmentation method and system based on data augmentation machine vision and electronic equipment | |
| CN113792809A (en) | Remote sensing picture classification method based on random semi-supervised feature extraction model | |
| Yildirim et al. | REGP: A NEW POOLING ALGORITHM FOR DEEP CONVOLUTIONAL NEURAL NETWORKS. | |
| CN115100509B (en) | Image identification method and system based on multi-branch block-level attention enhancement network | |
| CN111209886A (en) | Rapid pedestrian re-identification method based on deep neural network | |
| CN114445726B (en) | Sample library establishing method and device based on deep learning | |
| Shah et al. | A multi-blocked image classifier for deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |