CN113869136B - Semi-supervised polarization SAR image classification method based on multi-branch network - Google Patents

Semi-supervised polarization SAR image classification method based on multi-branch network Download PDF

Info

Publication number
CN113869136B
CN113869136B CN202111032840.2A CN202111032840A CN113869136B CN 113869136 B CN113869136 B CN 113869136B CN 202111032840 A CN202111032840 A CN 202111032840A CN 113869136 B CN113869136 B CN 113869136B
Authority
CN
China
Prior art keywords
network
image
classification
module
polsar
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111032840.2A
Other languages
Chinese (zh)
Other versions
CN113869136A (en
Inventor
李明
辛欣悦
张鹏
吴艳
徐大治
郑佳
胡欣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN202111032840.2A priority Critical patent/CN113869136B/en
Publication of CN113869136A publication Critical patent/CN113869136A/en
Application granted granted Critical
Publication of CN113869136B publication Critical patent/CN113869136B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a semi-supervised polarized SAR image classification method based on a multi-branch network, which comprises the following implementation steps of; constructing a test sample set, a marked training sample set and a unmarked training sample set; constructing a semi-supervised polarized SAR image classification model H based on a multi-branch network; performing iterative training on the polarized SAR image classification model H; and obtaining a classification result of the PolSAR image. In the process of training the image classification model, the two networks in the advanced feature extraction module respectively carry out advanced feature extraction on the marked training sample and the unmarked training sample, then classification in different modes is carried out through the three classification modules, and the MFB fusion module can carry out MFB fusion on each first advanced feature of the marked training sample and the second advanced feature of the corresponding position, so that the problems of over-fitting and redundancy in the prior art are effectively solved, and the precision of polarized SAR image classification is improved.

Description

Semi-supervised polarization SAR image classification method based on multi-branch network
Technical Field
The invention belongs to the technical field of image processing, relates to a polarized SAR image classification method, and in particular relates to a semi-supervised polarized SAR image classification method based on a multi-branch network. Can be used for agricultural development, ocean monitoring, urban planning, geological exploration and the like.
Background
The synthetic aperture radar (SYNTHETIC APERTURE RADAR, SAR) is insensitive to weather conditions and illumination conditions, and compared with optical remote sensing, the SAR is not influenced by factors such as weather, cloud layers and the like, and remote sensing data can be obtained all the day and the day. The polarized SAR (polarimetric synthetic aperture radar, polSAR) alternately transmits and receives radar signals in a horizontal polarization and vertical polarization mode, so that more complete and richer target information can be obtained, and the targets are more fully described. The classification of the PolSAR image aims at dividing the PolSAR image into different ground object categories according to the characteristic difference between classifying units in the PolSAR image, and plays an important role in aspects of agricultural development, ocean monitoring, urban planning, geological exploration and the like.
The traditional PolSAR image classification algorithm needs to set a specific algorithm for a specific target according to a great deal of experience and strong expertise, and is long in time consumption and difficult to popularize. In recent years, the PolSAR image classification method based on deep learning realizes the PolSAR image classification based on data driving, and the method can autonomously learn and extract the characteristics effective for classification from the data without manually selecting the characteristics, designing a classifier and strong professional knowledge. According to whether priori knowledge is needed in the classification process, the deep learning-based PolSAR image classification can be classified into three categories of supervised classification, unsupervised classification and semi-supervised classification. The semi-supervised classification method is a classification method combining supervised classification and unsupervised classification, can use marked data and unmarked data simultaneously, and can bring higher classification precision on the premise of reducing the workload of acquiring priori knowledge. For example, the self-training algorithm and the combined training algorithm are used for performing supervised training by using marked data as a training set to obtain a classifier, then the non-marked data is classified by using the classifier, a non-marked sample with high reliability and a predictive mark thereof are selected to be added into the training set according to a classification result, the scale of the training set is expanded, and the supervised training is performed again to obtain a new classifier.
For example, patent application with application publication number of CN112966779A, named as a PolSAR image semi-supervised classification method, discloses a PolSAR image semi-supervised classification method. According to the method, on the basis of a small number of marked training samples, a PolSAR image is classified by using a Wishare classifier, an SVM classifier and a CV-CNN model, majority voting is carried out on the classification result, a strong data set and a weak data set are generated, the strong data set is used as a pseudo tag, the weak data set is reclassified by using three classifiers, three classification results are integrated in a majority voting mode, and finally the strong data set is combined with the reclassification result to obtain a final classification result. The combined training algorithm adopted by the method fully utilizes the advantages of all classifiers, obtains higher classification precision, but still has the over-fitting problem caused by training a network by using a small amount of marked data and the redundancy problem caused by the multi-classifier fusion algorithm of majority voting, so that the classification precision of the PolSAR image is still lower.
Disclosure of Invention
The invention aims to solve the technical problem of low classification precision in the prior art by providing a semi-supervised polarized SAR image classification method based on a multi-branch network.
In order to achieve the above purpose, the technical scheme adopted by the invention comprises the following steps:
(1) Obtaining a test sample set D test and a labeled training sample set And a label-free training sample set
(1A) S PolSAR images P= { P s |1S less than or equal to C } containing C ground object categories L= { L c |1C less than or equal to C } are obtained, image block segmentation is carried out on each PolSAR image P s to obtain an image block set P '= { P s' |1S less than or equal to S,Each image block is then acquiredFeatures of (2)Wherein C is more than or equal to 2, L c represents the C-th ground object category, S is more than or equal to 100, P s represents the S-th PolSAR image, P s' represents a subset which corresponds to P s and contains V s image blocks,Representing the V s image block in P s', wherein V s is more than or equal to 1000;
(1b) Randomly selecting N PolSAR images in the PolSAR image set P as a test set P test＝{Ps1 |1 is less than or equal to s1 is less than or equal to N, and taking each image block in the image block set P s1' corresponding to each PolSAR image P s1 Features of (2)The test sample is used as a test sample set D test corresponding to P test, S-N PolSAR images remained in P are used as a training set P train＝{Ps2 |1 is less than or equal to S2 is less than or equal to S-N, and each image block in an image block set P s2' corresponding to each PolSAR image P s2 is obtainedClustering to obtainCluster marking of (c)Wherein the method comprises the steps ofB is the number of clusters;
(1c) Randomly selecting N l PolSAR images P l train＝{Ps21|1≤s21≤Nl from the training set P train, and for each image block in the image block set P s21' corresponding to each PolSAR image P s21 Carrying out true ground object category marking to obtainGround object mark of (2)Then willFeatures of (2)Ground object markClustering labelsForming marked training samples to obtain a marked training sample set corresponding to the P l train Wherein,
(1D) The rest S-N-N l PolSAR images in the training set P train are processedEach image block in the set of image blocks P s22' corresponding to each PolSAR image P s22 of (a)Features of (2)Clustering labelsForming a label-free training sample to obtainCorresponding unlabeled training sample set
(2) Constructing a semi-supervised polarized SAR image classification model H based on a multi-branch network:
constructing a semi-supervised polarized SAR image classification model H comprising an advanced feature extraction module and a multi-branch processing module which are sequentially cascaded, wherein:
The advanced feature extraction module comprises a first network and a second network which are arranged in parallel, wherein the first network and the second network comprise a convolution layer shared by a plurality of parameters, a plurality of batch normalization layers and a plurality of activation layers, the first network further comprises a first full-connection layer, and the second network further comprises a second full-connection layer with different parameters from those of the first full-connection layer;
The multi-branch processing module comprises an MFB fusion module, a first classification module, a second classification module and a third classification module which are arranged in parallel, wherein the output end of the MFB fusion module is cascaded with the third classification module; the MFB fusion module comprises a third full-connection layer, a matrix dot product module and a pooling layer which are sequentially cascaded; the first classification module comprises a fourth full connection layer and a Softmax activation layer which are cascaded; the second classification module comprises a fifth full connection layer and a Softmax activation layer which are cascaded, and the third classification module comprises a sixth full connection layer and a Softmax activation layer which are cascaded;
(3) Iterative training is carried out on a semi-supervised polarized SAR image classification model H based on a multi-branch network:
(3a) Initializing iteration times as I, wherein the maximum iteration times are I, I is more than or equal to 200, the weight parameter of an image classification model of the ith iteration as H i,Hi is omega i, and i=1, H i =H;
(3b) Will train the sample set from the marked M l marked training samples which are replaced and randomly selected, and from the unmarked training sample setM u unmarked training samples which are put back and randomly selected are used as the input of a semi-supervised polarized SAR image classification model H i, and a first network in the advanced feature extraction module respectively carries out advanced feature extraction on each marked training sample and each unmarked training sample to obtain a first advanced feature set of the marked training samplesAnd advanced feature set for unlabeled training samplesAt the same time, the second network performs advanced feature extraction on each marked training sample to obtain a second advanced feature set of the marked training samplesWherein the method comprises the steps ofAndRepresenting a first high-level feature of the mth 1 marked training samples output via the first network and a second high-level feature output via the second network, respectively,Representing the advanced features of the M 2 unmarked training samples output through the first network, wherein M l≤Nl×Vs,50≤Mu≤(S-N-Nl)×Vs is more than or equal to 30;
(3c) The first classification module classifies each high-level feature in the first high-level feature set F1 l of the marked training sample and the high-level feature set F1 u of the unmarked training sample to obtain a first prediction label set corresponding to F1 l And a second set of predictive labels corresponding to F1 u At the same time, the second classification module classifies each high-level feature of the second high-level feature set F2 l with the marked training sample to obtain a third prediction label set corresponding to F2 l The MFB fusion module fuses each first advanced feature in the first advanced feature set F1 l of the labeled training sampleSecond high-level features corresponding to positions of the second high-level feature set F2 l Performing MFB fusion, and performing third classification module pairAnd (3) withClassifying the fusion results of the (4) to obtain a fourth predictive labelThe fourth set of predictive labels corresponding to the fusion result of F1 l and F2 l is
(3D) Employing cross entropy loss function and training samples through each markedCorresponding first predictive labelClustering labelsCalculate the first loss value of H i By each unlabeled training sampleCorresponding second predictive labelClustering labelsCalculate the second loss value of H i By each marked training sampleCorresponding third predictive labelAnd ground object markCalculate the third loss value of H i By each marked training sampleCorresponding fourth predictive labelAnd ground object markCalculate the fourth loss value of H i And will beAndSum as total Loss value of H i Loss i:
(3e) Obtaining partial derivatives of Loss i on weight parameter omega i And by gradient descent methodUpdating the weight parameter omega i in a back propagation mode in H i;
(3f) Judging whether I is equal to or greater than I, if yes, obtaining a trained semi-supervised polarization SAR image classification model H * based on a multi-branch network, otherwise, enabling I to be equal to i+1, and executing the step (3 b);
(4) Obtaining a classification result of the PolSAR image:
(4a) Each test sample in the test sample set D test As the input of a trained semi-supervised polarized SAR image classification model H * based on a multi-branch network; the first network and the second network in the advanced feature extraction module are respectively opposite to each otherAdvanced feature extraction is carried out to obtainCorresponding first advanced featuresAnd a second advanced feature
(4B) MFB fusion module pair test sampleCorresponding first advanced featuresAnd a second advanced featurePerforming MFB fusion, and performing third classification module pairAnd (3) withClassifying the fusion results of the test samplePredictive tag of (a)The V s prediction labels are the classification result of the corresponding PolSAR image P s1.
Compared with the prior art, the invention has the following advantages:
1. the multi-branch processing module in the polarized SAR image classification model constructed by the invention comprises a first classification module, a second classification module and a third classification module, wherein in the process of training the image classification model, two networks in the advanced feature extraction module respectively carry out advanced feature extraction on marked training samples and unmarked training samples, and then classification in different modes is carried out through the three classification modules, so that the problem that the classification model is over-fitted due to the fact that only a small amount of marked data is used in the prior art is avoided, and the classification precision of the PolSAR image is effectively improved.
2. The polarization SAR image classification model constructed by the method also comprises an MFB fusion module, in the process of training the image classification model, the MFB fusion module carries out MFB fusion on each first high-level feature of the marked training sample and the second high-level feature of the corresponding position, and then the fusion result is classified by a third classification module.
Drawings
FIG. 1 is a flow chart of an implementation of the present invention;
FIG. 2 is a schematic diagram of the overall structure of a polarized SAR image classification model constructed in accordance with the present invention;
FIG. 3 is a schematic diagram of an advanced feature extraction module employed in an embodiment of the invention;
Fig. 4 is a schematic diagram of a multi-branch processing module constructed in accordance with the present invention.
Detailed Description
The invention is described in further detail below with reference to the attached drawings and specific examples:
referring to fig. 1, the present invention includes the steps of:
Step 1) obtaining a test sample set D test and a labeled training sample set And a label-free training sample set
Step 1 a) obtains S PolSAR images P= { P s |1S less than or equal to S } containing C ground object categories L= { L c |1C less than or equal to C }, and performs image block segmentation on each PolSAR image P s to obtain an image block set P '= { P s' |1S less than or equal to S },Each image block is then acquiredFeatures of (2)Wherein C is more than or equal to 2, L c represents the C-th ground object category, S is more than or equal to 100, P s represents the S-th PolSAR image, P s' represents a subset which corresponds to P s and contains V s image blocks,Representing the V s image block in P s', wherein V s is more than or equal to 1000; in this embodiment, s=200, c= 8,V s =1500;
Acquiring each image block Features of (2)The implementation steps are as follows: acquiring each image blockIs of the horizontal polarization component of (2)Vertical polarization componentAnd cross-polarized componentI.e. scattering matrixAnd is opposite toPauli decomposition is carried out to obtain each image blockFeatures of (2)
Wherein [ (T ] represents a transpose operation).
Step 1 b) randomly selecting N PolSAR images in the PolSAR image set P as a test set P test＝{Ps1 |1 is less than or equal to s1 is less than or equal to N, and taking each image block in the image block set P s1' corresponding to each PolSAR image P s1 Features of (2)The test sample is used as a test sample set D test corresponding to P test, S-N PolSAR images remained in P are used as a training set P train＝{Ps2 |1 is less than or equal to S2 is less than or equal to S-N, and each image block in an image block set P s2' corresponding to each PolSAR image P s2 is obtainedClustering to obtainCluster marking of (c)Wherein the method comprises the steps ofB is the number of clusters; in this embodiment, n=40, b=10,
Because selecting enough training samples avoids over-fitting the network, in this embodiment, the ratio of the number of samples in the selected test sample set to the number of samples in the training sample set to the total number of samples is 20% and 80%, respectively;
For each image block in the set of image blocks P s2' corresponding to each PolSAR image P s2 Clustering is carried out, and the implementation steps are as follows:
Step 1b 1) acquiring each image block Is of the horizontal polarization component of (2)Vertical polarization componentAnd cross-polarized componentI.e. scattering matrixAnd is opposite toPauli decomposition is carried out to obtain a three-dimensional Pauli feature vectorAnd pass throughAnd conjugate transpose thereofConstructionCorresponding coherence matrix
Wherein, [ ] H represents a conjugate transpose operation, [ ] * represents a conjugate operation;
Step 1B 2) initializing iteration times to be Iter, wherein the maximum iteration times are Iter, iter is more than or equal to 10, and randomly dividing all image blocks in an image block set P s2' corresponding to each acquired PolSAR image P s2 into B disjoint subsets Each subset ofCorresponding to a markAnd let iter=1; in this embodiment, item=20;
Step 1b 3) computing each subset A corresponding mean-coherence matrix Σ b, and calculating Σ b and each image blockCorresponding coherence matrixIs a wishart distance of (2)
Wherein,Representing subsetsThe corresponding coherence matrix of the kth image block p k in (a), K b representing the subsetSigma (-) represents the summation operation, tr (-) represents the trace operation, and [ (35) -1 represents the inversion operation;
step 1b 4) each image block Into the subset with the smallest distance to its wishart,Cluster marking of (c)Marking the subset to which the item belongs, judging whether the item is equal to or more than Iter, if so, obtaining each image blockCluster marking of (c)Otherwise let iter=iter+1 and perform step 1b 3).
Because the coherence matrix in the PolSAR data accords with complex wishart distribution, the clustering method using the wishart distance accords with the PolSAR image classification scene.
Step 1 c) randomly selecting N l PolSAR images P l train＝{Ps21|1≤s21≤Nl from the training set P train, and for each image block in the image block set P s21' corresponding to each PolSAR image P s21 Carrying out true ground object category marking to obtainGround object mark of (2)Then willFeatures of (2)Ground object markClustering labelsForming marked training samples to obtain a marked training sample set corresponding to the P l train Wherein,In this embodiment, N l =48;
In the practical application scene, the real ground object marking is performed on the training samples, so that in the embodiment, the proportion of the number of samples in the marked training sample set and the unmarked training sample set to the total training sample number is 30% and 70% respectively;
step 1 d) remaining S-N-N l PolSAR images in the training set P train Each image block in the set of image blocks P s22' corresponding to each PolSAR image P s22 of (a)Features of (2)Clustering labelsForming a label-free training sample to obtainCorresponding unlabeled training sample set
Step 2) constructing a semi-supervised polarized SAR image classification model H based on a multi-branch network:
constructing a semi-supervised polarized SAR image classification model H comprising an advanced feature extraction module and a multi-branch processing module which are sequentially cascaded, wherein the structure of the semi-supervised polarized SAR image classification model H is shown in figure 2;
Referring to fig. 3, the advanced feature extraction module includes a first network and a second network arranged in parallel, where the first network and the second network each include a convolution layer with multiple parameter sharing, multiple batch normalization layers, and multiple activation layers, the first network further includes a first fully-connected layer, and the second network further includes a second fully-connected layer with different parameters from the first fully-connected layer; in this embodiment, the number of convolution layers in the first network and the second network included in the advanced feature extraction module is 2, the size of the convolution kernel of the first convolution layer is 3×3, the convolution step length is 1, the number of the convolution kernels is 16, the size of the convolution kernel of the second convolution layer is 5×5, the convolution step length is 1, and the number of the convolution kernels is 32; the number of batch normalization layers is 2; the active layers adopt ReLu active layers, and the number of ReLu active layers in the first network and the second network is 2; the number of the first full-connection layer neurons is 64, and the number of the second full-connection layer neurons is 32; the specific structure of the first network is as follows: first convolution layer- & gt first batch normalization layer- & gt first ReLu activation layer- & gt second convolution layer- & gt second batch normalization layer- & gt second ReLu activation layer- & gt first full connection layer; the second network has the same basic structure as the first network, and the first full connection layer in the first network is replaced by the second full connection layer only.
Referring to fig. 4, the multi-branch processing module includes an MFB fusion module, and a first classification module, a second classification module, and a third classification module arranged in parallel, where an output end of the MFB fusion module is cascaded with the third classification module; the MFB fusion module comprises a third full-connection layer, a matrix dot product module and a pooling layer which are sequentially cascaded; the first classification module comprises a fourth full connection layer and a Softmax activation layer which are cascaded; the second classification module comprises a fifth full connection layer and a Softmax activation layer which are cascaded, and the third classification module comprises a sixth full connection layer and a Softmax activation layer which are cascaded; in this embodiment, the number of neurons of the third full connection layer in the MFB fusion module included in the multi-branch processing module is 128; the number of neurons of the fourth full-connection layer contained in the first classification module is equal to the number B of clustering each image block in the step (1B); the number of neurons of the fifth full-connection layer contained in the second classification module and the sixth full-connection layer contained in the third classification module is equal to the number C of the ground object categories.
Step 3) carrying out iterative training on a semi-supervised polarized SAR image classification model H based on a multi-branch network:
Step 3 a), initializing iteration times as I, wherein the maximum iteration times are I, I is more than or equal to 200, the weight parameter of an image classification model of the ith iteration as H i,Hi is omega i, and i=1, H i =H; wherein, in the present embodiment, i=300;
Step3 b) training the sample set from the marked M l marked training samples which are replaced and randomly selected, and from the unmarked training sample setM u unmarked training samples which are put back and randomly selected are used as the input of a semi-supervised polarized SAR image classification model H i, and a first network in the advanced feature extraction module respectively carries out advanced feature extraction on each marked training sample and each unmarked training sample to obtain a first advanced feature set of the marked training samplesAnd advanced feature set for unlabeled training samplesAt the same time, the second network performs advanced feature extraction on each marked training sample to obtain a second advanced feature set of the marked training samplesWherein the method comprises the steps ofAndRepresenting a first high-level feature of the mth 1 marked training samples output via the first network and a second high-level feature output via the second network, respectively,Representing the advanced features of the M 2 unmarked training samples output through the first network, wherein M l≤Nl×Vs,50≤Mu≤(S-N-Nl)×Vs is more than or equal to 30; in this embodiment, M l＝50,Mu =60;
Step 3 c) the first classification module classifies each of the first advanced features of the marked training sample set F1 l and the unmarked training sample set F1 u to obtain a first set of predictive labels corresponding to F1 l And a second set of predictive labels corresponding to F1 u At the same time, the second classification module classifies each high-level feature of the second high-level feature set F2 l with the marked training sample to obtain a third prediction label set corresponding to F2 l The MFB fusion module fuses each first advanced feature in the first advanced feature set F1 l of the labeled training sampleSecond high-level features corresponding to positions of the second high-level feature set F2 l Performing MFB fusion, and performing third classification module pairAnd (3) withClassifying the fusion results of the (4) to obtain a fourth predictive labelThe fourth set of predictive labels corresponding to the fusion result of F1 l and F2 l is
The classification in different modes is carried out through the first classification module, the second classification module and the third classification module, so that the problem that the classification model is over-fitted due to the fact that only a small amount of marked data is used in the prior art is avoided, and the classification precision of the PolSAR image is effectively improved.
The MFB fusion module fuses each first advanced feature in the first advanced feature set F1 l of the labeled training sampleSecond high-level features corresponding to positions of the second high-level feature set F2 l The MFB fusion is carried out, and the implementation steps are as follows: third full connection layer pair first advanced features of MFB moduleAnd second advanced featuresRespectively through dimension conversion to obtain high-grade features with the same dimensionAnd advanced featuresMatrix dot product module pairAndPerforming dot product, and pooling the dot product result by pooling layer to obtainAnd (3) withIs a fusion result of (2).
The MFB fusion method avoids redundancy problem caused by multi-classifier fusion algorithm of majority voting adopted in the prior art, and further improves classification accuracy of the PolSAR image.
Step 3 d) employs a cross entropy loss function and training samples through each of the markedCorresponding first predictive labelClustering labelsCalculate the first loss value of H i By each unlabeled training sampleCorresponding second predictive labelClustering labelsCalculate the second loss value of H i By each marked training sampleCorresponding third predictive labelAnd ground object markCalculate the third loss value of H i By each marked training sampleCorresponding fourth predictive labelAnd ground object markCalculate the fourth loss value of H i And will beAndSum as total Loss value of H i Loss i:
First loss value Second loss valueThird loss valueAnd fourth loss valueThe calculation formulas are respectively as follows:
where Σ (-) represents the summation operation and In (-) represents the log-taking operation based on the natural constant e.
Step 3 e) obtaining the partial derivative of Loss i on weight parameter omega i And by gradient descent methodUpdating the weight parameter omega i in a back propagation mode in H i;
Updating the weight parameter omega i, wherein the updating formula is as follows:
Where ω i' represents the updated result of ω i, η represents the learning rate, The derivation operation is represented, and in this example, the learning rate η=0.001.
Step 3 f) judging whether I is more than or equal to I, if yes, obtaining a trained semi-supervised polarized SAR image classification model H * based on a multi-branch network, otherwise, enabling I to be equal to i+1, and executing step 3 b);
Step 4) obtaining a classification result of the PolSAR image:
(4a) Each test sample in the test sample set D test As the input of a trained semi-supervised polarized SAR image classification model H * based on a multi-branch network; the first network and the second network in the advanced feature extraction module are respectively opposite to each otherAdvanced feature extraction is carried out to obtainCorresponding first advanced featuresAnd a second advanced feature
(4B) MFB fusion module pair test sampleCorresponding first advanced featuresAnd a second advanced featurePerforming MFB fusion, and performing third classification module pairAnd (3) withClassifying the fusion results of the test samplePredictive tag of (a)The V s prediction labels are the classification result of the corresponding PolSAR image P s1.

Claims (7)

1. A semi-supervised polarized SAR image classification method based on a multi-branch network is characterized by comprising the following steps:
(1) Obtaining a test sample set D test and a labeled training sample set And a label-free training sample set
(1A) S PolSAR images P= { P s |1S less than or equal to C } containing C ground object categories L= { L c |1C less than or equal to C } are obtained, image block segmentation is carried out on each PolSAR image P s to obtain an image block set P '= { P s' |1S less than or equal to S,Each image block is then acquiredFeatures of (2)Wherein C is more than or equal to 2, L c represents the C-th ground object category, S is more than or equal to 100, P s represents the S-th PolSAR image, P s' represents a subset which corresponds to P s and contains V s image blocks,Representing the V s image block in P s', wherein V s is more than or equal to 1000;
(1b) Randomly selecting N PolSAR images in the PolSAR image set P as a test set P test＝{Ps1 |1 is less than or equal to s1 is less than or equal to N, and taking each image block in the image block set P s1' corresponding to each PolSAR image P s1 Features of (2)The test sample is used as a test sample set D test corresponding to P test, S-N PolSAR images remained in P are used as a training set P train＝{Ps2 |1 is less than or equal to S2 is less than or equal to S-N, and each image block in an image block set P s2' corresponding to each PolSAR image P s2 is obtainedClustering to obtainCluster marking of (c)Wherein the method comprises the steps ofB is the number of clusters;
(1c) Randomly selecting N l PolSAR images from the training set P train And for each image block in the set of image blocks P s21' corresponding to each PolSAR image P s21 Carrying out true ground object category marking to obtainGround object mark of (2)Then willFeatures of (2)Ground object markClustering labelsForming a marked training sample to obtainCorresponding labeled training sample setWherein,
(1D) The rest S-N-N l PolSAR images in the training set P train are processedEach image block in the set of image blocks P s22' corresponding to each PolSAR image P s22 of (a)Features of (2)Clustering labelsForming a label-free training sample to obtainCorresponding unlabeled training sample set
(2) Constructing a semi-supervised polarized SAR image classification model H based on a multi-branch network:
constructing a semi-supervised polarized SAR image classification model H comprising an advanced feature extraction module and a multi-branch processing module which are sequentially cascaded, wherein:
The advanced feature extraction module comprises a first network and a second network which are arranged in parallel, wherein the first network and the second network comprise a convolution layer shared by a plurality of parameters, a plurality of batch normalization layers and a plurality of activation layers, the first network further comprises a first full-connection layer, and the second network further comprises a second full-connection layer with different parameters from those of the first full-connection layer;
The multi-branch processing module comprises an MFB fusion module, a first classification module, a second classification module and a third classification module which are arranged in parallel, wherein the output end of the MFB fusion module is cascaded with the third classification module; the MFB fusion module comprises a third full-connection layer, a matrix dot product module and a pooling layer which are sequentially cascaded; the first classification module comprises a fourth full connection layer and a Softmax activation layer which are cascaded; the second classification module comprises a fifth full connection layer and a Softmax activation layer which are cascaded, and the third classification module comprises a sixth full connection layer and a Softmax activation layer which are cascaded;
(3) Iterative training is carried out on a semi-supervised polarized SAR image classification model H based on a multi-branch network:
(3a) Initializing iteration times as I, wherein the maximum iteration times are I, I is more than or equal to 200, the weight parameter of an image classification model of the ith iteration as H i,Hi is omega i, and i=1, H i =H;
(3b) Will train the sample set from the marked M l marked training samples which are replaced and randomly selected, and from the unmarked training sample setM u unmarked training samples which are put back and randomly selected are used as the input of a semi-supervised polarized SAR image classification model H i, and a first network in the advanced feature extraction module respectively carries out advanced feature extraction on each marked training sample and each unmarked training sample to obtain a first advanced feature set of the marked training samplesAnd advanced feature set for unlabeled training samplesAt the same time, the second network performs advanced feature extraction on each marked training sample to obtain a second advanced feature set of the marked training samplesWherein the method comprises the steps ofAndRepresenting a first high-level feature of the mth 1 marked training samples output via the first network and a second high-level feature output via the second network, respectively,Representing the advanced features of the M 2 unmarked training samples output through the first network, wherein M l≤Nl×Vs,50≤Mu≤(S-N-Nl)×Vs is more than or equal to 30;
(3c) The first classification module classifies each high-level feature in the first high-level feature set F1 l of the marked training sample and the high-level feature set F1 u of the unmarked training sample to obtain a first prediction label set corresponding to F1 l And a second set of predictive labels corresponding to F1 u At the same time, the second classification module classifies each high-level feature of the second high-level feature set F2 l with the marked training sample to obtain a third prediction label set corresponding to F2 l The MFB fusion module fuses each first advanced feature in the first advanced feature set F1 l of the labeled training sampleSecond high-level features corresponding to positions of the second high-level feature set F2 l Performing MFB fusion, and performing third classification module pairAnd (3) withClassifying the fusion results of the (4) to obtain a fourth predictive labelThe fourth set of predictive labels corresponding to the fusion result of F1 l and F2 l is
(3D) Employing cross entropy loss function and training samples through each markedCorresponding first predictive labelClustering labelsCalculate the first loss value of H i By each unlabeled training sampleCorresponding second predictive labelClustering labelsCalculate the second loss value of H i By each marked training sampleCorresponding third predictive labelAnd ground object markCalculate the third loss value of H i By each marked training sampleCorresponding fourth predictive labelAnd ground object markCalculate the fourth loss value of H i And will beAndSum as total Loss value of H i Loss i:
(3e) Obtaining partial derivatives of Loss i on weight parameter omega i And by gradient descent methodUpdating the weight parameter omega i in a back propagation mode in H i;
(3f) Judging whether I is equal to or greater than I, if yes, obtaining a trained semi-supervised polarization SAR image classification model H * based on a multi-branch network, otherwise, enabling I to be equal to i+1, and executing the step (3 b);
(4) Obtaining a classification result of the PolSAR image:
(4a) Each test sample in the test sample set D test As the input of a trained semi-supervised polarized SAR image classification model H * based on a multi-branch network; the first network and the second network in the advanced feature extraction module are respectively opposite to each otherAdvanced feature extraction is carried out to obtainCorresponding first advanced featuresAnd a second advanced feature
(4B) MFB fusion module pair test sampleCorresponding first advanced featuresAnd a second advanced featurePerforming MFB fusion, and performing third classification module pairAnd (3) withClassifying the fusion results of the test samplePredictive tag of (a)The V s prediction labels are the classification result of the corresponding PolSAR image P s1.
2. The multi-branch network-based semi-supervised polarized SAR image classification method according to claim 1, wherein each image block is acquired in step (1 a)Features of (2)The implementation steps are as follows:
Acquiring each image block Is of the horizontal polarization component of (2)Vertical polarization componentAnd cross-polarized componentI.e. scattering matrixAnd is opposite toPauli decomposition is carried out to obtain each image blockFeatures of (2)
Wherein [ (T ] represents a transpose operation).
3. The multi-branch network-based semi-supervised polarized SAR image classification method according to claim 1, wherein each image block in the set of image blocks P s2' corresponding to each PolSAR image P s2 in step (1 b)Clustering is carried out, and the implementation steps are as follows:
(1b1) Acquiring each image block Is of the horizontal polarization component of (2)Vertical polarization componentAnd cross-polarized componentI.e. scattering matrixAnd is opposite toPauli decomposition is carried out to obtain a three-dimensional Pauli feature vectorAnd pass throughAnd conjugate transpose thereofConstructionCorresponding coherence matrix
Wherein, [ ] H represents a conjugate transpose operation, [ ] * represents a conjugate operation;
(1b2) Initializing iteration times to be Iter, and dividing all image blocks in an image block set P s2' corresponding to each acquired PolSAR image P s2 into B disjoint subsets at random, wherein the maximum iteration times are Iter and not less than 10 Each subset ofCorresponding to a markAnd let iter=1;
(1b3) Calculate each subset A corresponding mean-coherence matrix Σ b, and calculating Σ b and each image blockCorresponding coherence matrixIs a wishart distance of (2)
Wherein,Representing subsetsThe corresponding coherence matrix of the kth image block p k in (a), K b representing the subsetSigma (-) represents the summation operation, tr (-) represents the trace operation, and [ (35) -1 represents the inversion operation;
(1b4) Each image block Into the subset with the smallest distance to its wishart,Cluster marking of (c)Marking the subset to which the item belongs, judging whether the item is equal to or more than Iter, if so, obtaining each image blockCluster marking of (c)Otherwise, let iter=iter+1 and perform step (1 b 3).
4. The multi-branch network-based semi-supervised polarized SAR image classification method according to claim 1, wherein said advanced feature extraction module and multi-branch processing module of step (2), wherein:
The number of convolution layers in the first network and the second network contained in the advanced feature extraction module is 2, the size of the convolution kernel of the first convolution layer is 3 multiplied by 3, the convolution step length is 1, the number of the convolution kernels is 16, the size of the convolution kernel of the second convolution layer is 5 multiplied by 5, the convolution step length is 1, and the number of the convolution kernels is 32; the number of batch normalization layers is 2; the active layers adopt ReLu active layers, and the number of ReLu active layers in the first network and the second network is 2; the number of the first full-connection layer neurons is 64, and the number of the second full-connection layer neurons is 32; the specific structure of the first network is as follows: first convolution layer- & gt first batch normalization layer- & gt first ReLu activation layer- & gt second convolution layer- & gt second batch normalization layer- & gt second ReLu activation layer- & gt first full connection layer; the basic structure of the second network is the same as that of the first network, and the first full-connection layer in the first network is replaced by the second full-connection layer only;
the number of neurons of a third full connection layer in the MFB fusion module contained in the multi-branch processing module is 128; the number of neurons of the fourth full-connection layer contained in the first classification module is equal to the number B of clustering each image block in the step (1B); the number of neurons of the fifth full-connection layer contained in the second classification module and the sixth full-connection layer contained in the third classification module is equal to the number C of the ground object categories.
5. The multi-branch network based semi-supervised polarized SAR image classification method according to claim 1, wherein said MFB fusion module of step (3 c) is configured to, for each first high level feature in the first high level feature set F1 l of the labeled training samplesSecond high-level features corresponding to positions of the second high-level feature set F2 l The MFB fusion is carried out, and the implementation steps are as follows:
third full connection layer pair first advanced features of MFB module And second advanced featuresRespectively through dimension conversion to obtain high-grade features with the same dimensionAnd advanced featuresMatrix dot product module pairAndPerforming dot product, and pooling the dot product result by pooling layer to obtainAnd (3) withIs a fusion result of (2).
6. The multi-branch network-based semi-supervised polarimetric SAR image classification method according to claim 1, wherein said first loss value in step (3 d)Second loss valueThird loss valueAnd fourth loss valueThe calculation formulas are respectively as follows:
where Σ (-) represents the summation operation and In (-) represents the log-taking operation based on the natural constant e.
7. The multi-branch network-based semi-supervised polarimetric SAR image classification method according to claim 1, wherein the updating of the weight parameter ω i in step (3 e) is performed by:
Where ω i' represents the updated result of ω i, η represents the learning rate, Representing a derivative operation.
CN202111032840.2A 2021-09-03 2021-09-03 Semi-supervised polarization SAR image classification method based on multi-branch network Active CN113869136B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111032840.2A CN113869136B (en) 2021-09-03 2021-09-03 Semi-supervised polarization SAR image classification method based on multi-branch network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111032840.2A CN113869136B (en) 2021-09-03 2021-09-03 Semi-supervised polarization SAR image classification method based on multi-branch network

Publications (2)

Publication Number Publication Date
CN113869136A CN113869136A (en) 2021-12-31
CN113869136B true CN113869136B (en) 2024-07-02

Family

ID=78989516

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111032840.2A Active CN113869136B (en) 2021-09-03 2021-09-03 Semi-supervised polarization SAR image classification method based on multi-branch network

Country Status (1)

Country Link
CN (1) CN113869136B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114998840B (en) * 2022-07-18 2022-11-29 成都东方天呈智能科技有限公司 Mouse target detection method based on deep cascade supervised learning

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107563422A (en) * 2017-08-23 2018-01-09 西安电子科技大学 A kind of polarization SAR sorting technique based on semi-supervised convolutional neural networks
CN111695466A (en) * 2020-06-01 2020-09-22 西安电子科技大学 Semi-supervised polarization SAR terrain classification method based on feature mixup

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107563422A (en) * 2017-08-23 2018-01-09 西安电子科技大学 A kind of polarization SAR sorting technique based on semi-supervised convolutional neural networks
CN111695466A (en) * 2020-06-01 2020-09-22 西安电子科技大学 Semi-supervised polarization SAR terrain classification method based on feature mixup

Also Published As

Publication number Publication date
CN113869136A (en) 2021-12-31

Similar Documents

Publication Publication Date Title
CN108388927B (en) Small sample polarization SAR terrain classification method based on deep convolution twin network
CN111860612B (en) Unsupervised hyperspectral image hidden low-rank projection learning feature extraction method
CN106355151B (en) A kind of three-dimensional S AR images steganalysis method based on depth confidence network
CN110309868A (en) In conjunction with the hyperspectral image classification method of unsupervised learning
CN112052754B (en) Polarization SAR image ground object classification method based on self-supervision characterization learning
CN107491734B (en) Semi-supervised polarimetric SAR image classification method based on multi-core fusion and space Wishart LapSVM
CN112347970B (en) Remote sensing image ground object identification method based on graph convolution neural network
CN105989336B (en) Scene recognition method based on deconvolution deep network learning with weight
CN109581339B (en) Sonar identification method based on automatic adjustment self-coding network of brainstorming storm
Mussina et al. Multi-modal data fusion using deep neural network for condition monitoring of high voltage insulator
Qayyum et al. Designing deep CNN models based on sparse coding for aerial imagery: a deep-features reduction approach
CN111738052B (en) Multi-feature fusion hyperspectral remote sensing ground object classification method based on deep learning
Li et al. An aerial image segmentation approach based on enhanced multi-scale convolutional neural network
CN112149612A (en) Marine organism recognition system and recognition method based on deep neural network
CN113869136B (en) Semi-supervised polarization SAR image classification method based on multi-branch network
CN116152678A (en) Marine disaster-bearing body identification method based on twin neural network under small sample condition
CN114973019A (en) Deep learning-based geospatial information change detection classification method and system
CN113344069B (en) Image classification method for unsupervised visual representation learning based on multi-dimensional relation alignment
CN114782752A (en) Small sample image grouping classification method and device based on self-training
CN114329031A (en) Fine-grained bird image retrieval method based on graph neural network and deep hash
Wang et al. R2-trans: Fine-grained visual categorization with redundancy reduction
CN117611838A (en) Multi-label image classification method based on self-adaptive hypergraph convolutional network
Liu et al. Integration transformer for ground-based cloud image segmentation
CN111401519B (en) Deep neural network unsupervised learning method based on similarity distance in object and between objects
CN115329821A (en) Ship noise identification method based on pairing coding network and comparison learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant