CN110263845B - SAR image change detection method based on semi-supervised countermeasure depth network - Google Patents

SAR image change detection method based on semi-supervised countermeasure depth network Download PDF

Info

Publication number
CN110263845B
CN110263845B CN201910527007.1A CN201910527007A CN110263845B CN 110263845 B CN110263845 B CN 110263845B CN 201910527007 A CN201910527007 A CN 201910527007A CN 110263845 B CN110263845 B CN 110263845B
Authority
CN
China
Prior art keywords
layer
network
sample
training
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910527007.1A
Other languages
Chinese (zh)
Other versions
CN110263845A (en
Inventor
王英华
杨振东
王剑
刘宏伟
秦庆喜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN201910527007.1A priority Critical patent/CN110263845B/en
Publication of CN110263845A publication Critical patent/CN110263845A/en
Application granted granted Critical
Publication of CN110263845B publication Critical patent/CN110263845B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology

Abstract

The invention discloses a SAR image change detection method based on a semi-supervised countermeasure depth network, which mainly solves the problems that the detection effect false alarm rate is high and the detection area is inaccurate when the label data are less in the existing change detection technology. The scheme is as follows: 1) Calculating a logarithmic ratio difference map of the two images by using the two-time-phase SAR image data; 2) Extracting training samples and test samples from the two-phase SAR image and the difference image; 3) Constructing a change detection dual network and two discrimination networks; 4) Performing supervision training by using the labeled data, performing countermeasure training and collaborative training by using the unlabeled data, and obtaining a trained detection network; 5) And inputting the test data into a trained change detection network to obtain a change detection result. The invention combines a large amount of non-tag data to extract the change detection characteristics of the separability, improves the generalization performance of the supervision training model when the labeled training sample is insufficient, and can be used for SAR image change detection.

Description

SAR image change detection method based on semi-supervised countermeasure depth network
Technical Field
The invention belongs to the technical field of radar image processing, and particularly relates to a change detection method of SAR images, which can be used for disaster monitoring, land investigation and target investigation.
Background
The SAR system is less influenced by weather conditions and illumination conditions, and can realize all-weather earth observation in all weather, so that the change detection of multi-time-phase SAR data is an important means for analyzing the change of the earth surface, and is widely applied to disaster monitoring, land investigation and target investigation.
The traditional SAR change detection methods are of three types: the first type is an unsupervised change detection method, such as a change detection method based on principal component analysis and k-means clustering proposed by Celik et al; SAR variation detection method based on image fusion and fuzzy clustering proposed by Gong et al; the Yan Wang et al proposes a SIFT key point detection and region information-based change detection method. The second type is a supervised change detection method, such as the kernel-based change detection method proposed by Camps-Valls et al, which uses tagged data as training samples; yuLi et al, a change detection method based on matching pursuits. The third class is a semi-supervised change detection method, such as a SAR image change detection method based on a neighborhood clustering kernel proposed by Lu Jia and the like; the SAR image semi-supervised change detection algorithm based on random fields and maximum entropy is proposed by Linan et al.
Among the above three methods, the unsupervised method does not need to use labeled data, so it is a mainstream method in the field of change detection, but because of lack of supervision and guidance of labeled data, the detection result of the method is generally quite different from the real change area, and the false alarm is quite large. For the supervised method, the supervised method can obtain good results in a scene with a large amount of tagged data, but in a real scene, the cost of acquiring the tagged data is large, that is to say, the amount of tagged data is small in general, and in this case, the effect of the supervised method is poor, and the generalization performance of the model is poor. Compared with the supervised and unsupervised methods, the semi-supervised change detection method can be combined with a small amount of tagged data and a large amount of untagged data to learn together, the characteristic of separability is extracted, and the detection performance is improved. The existing semi-supervised SAR image change detection method is generally based on a semi-supervised method in traditional machine learning, the input characteristics of a model often need to be designed manually, all information of original data cannot be effectively utilized, and high false alarm rate and low detection precision are caused, so that the performance of the method is limited.
Disclosure of Invention
Aiming at the defects of the three existing SAR change detection methods, the invention provides an SAR image change detection method based on a semi-supervised countermeasure depth network, so that the detection precision is improved by combining a large amount of unlabeled data under the condition that the sample size of labeled data is small, and the false alarm rate is reduced.
The technical scheme of the invention is as follows: firstly, a small amount of labeled samples and a large amount of unlabeled samples are extracted by utilizing a sliding window model, then the labeled samples and the unlabeled samples are used for training a deep neural network model together, after the model converges, the trained neural network is applied to test data, and a final change detection mark graph is obtained, and the implementation steps comprise the following steps:
(1) Calculating a logarithmic ratio difference graph K of the two images by using the two-phase SAR image data;
(2) Extracting training samples and test samples on the two-time-phase SAR image and the difference image in a sliding window mode, randomly selecting 4% of the training samples as labeled training samples, and taking the rest as unlabeled training samples;
(3) Building a training network model:
(3a) Setting SAR change detection dual network ψ 1 And psi is 2 ：
Each network is composed of six layers, wherein the first four layers are sharing layers, namely the first layer is a full-connection layer L 1 The second layer is an activation function layer ReLU and the third layer is a full connection layer L 2 The fourth layer is an activation function layer ReLU, and the fifth and sixth are non-shared layers, wherein:
first network ψ 1 The fifth layer of (2) is a full connection layer L 13 The sixth layer is Softmax classifier layer S 11 ，
Second network ψ 2 The fifth layer of (2) is a full connection layer L 23 The sixth layer is Softmax classifier layer S 21 ；
(3b) Setting two discrimination networks
Figure BDA0002098559190000021
And->
Figure BDA0002098559190000022
The two discrimination networks are identical and are composed of six layers, namely, a first layer is a full-connection layer, a second layer is an activation function layer ReLU, a third layer is a full-connection layer, a fourth layer is an activation function layer, a fifth layer is a full-connection layer, a sixth layer is a Softmax classifier layer,
(3c) Connecting the dual network with two discrimination networks, i.e. the first discrimination network
Figure BDA0002098559190000023
Connected to a first detection network ψ 1 After that, the second discrimination network->
Figure BDA0002098559190000024
Connected to a second detection network ψ 2 Then, forming a training network model;
(6) Inputting training sample data into the constructed training network model (3), and sequentially performing supervised training with label data, countermeasure training without label data and collaborative training to obtain a trained change detection network ψ;
(7) And inputting the test sample data into a trained change detection network ψ for detection to obtain a change detection result of the SAR image.
Compared with the prior art, the invention has the following advantages:
1) According to the invention, the two-time phase SAR is utilized, so that the change detection characteristic of the separability can be extracted by combining a large amount of non-tag data under the condition that the sample size of the tagged sample is small, and the change detection performance is improved;
2) The invention utilizes the advantage of deep learning on classification tasks, and promotes the mutual promotion of two networks in the training process by combining a double-network structure with the countermeasure training and the cooperative training, thereby finally promoting the performance of change detection.
Drawings
FIG. 1 is a general flow chart of an implementation of the present invention;
FIG. 2 is a schematic diagram of constructing an input sample in the present invention;
FIG. 3 is a graph of experimental data;
fig. 4 is a graph showing the result of the change detection of the SAR image shown in fig. 3 by the present invention and the conventional method.
Detailed Description
The embodiments and effects of the present invention will be described in detail below with reference to the attached drawings:
referring to fig. 1, the implementation steps of the present invention are as follows:
and step 1, calculating a corresponding logarithmic ratio difference map according to the two-phase SAR image.
Calculating a corresponding logarithmic ratio difference graph K by using the two-phase SAR image data:
Figure BDA0002098559190000031
wherein I is 1 For the first time phase diagram of the original SAR image, I 2 Is the second time-phase diagram of the original SAR image.
And 2, extracting training samples and test samples from the two-time-phase SAR image and the difference image in a sliding window mode.
Referring to fig. 2, this step is implemented as follows:
2.1 Setting the size of the sliding window as N multiplied by N, setting the center of the sliding window as (i, j), for the training sample, performing sliding window selection image blocks in a designated area, and for the test sample, performing sliding window selection image blocks on the whole image;
2.2 Image I) in two phases 1 And I 2 In (i, j), image blocks of size N×N are selected as the center
Figure BDA0002098559190000032
And->
Figure BDA0002098559190000033
Will->
Figure BDA0002098559190000034
And->
Figure BDA0002098559190000035
Splicing together along a first dimension to obtain a sample first channel with the size of 2NxN;
2.3 Corresponding to two-phase image I 1 And I 2 A middle pixel point (i, j), and selecting an image block centered on (i, j) in the difference image K as a mark
Figure BDA0002098559190000036
The size of the image block is N, calculating +.>
Figure BDA0002098559190000037
Extending m into a matrix of size 2N x N as a sample second channel;
2.4 When a training sample is selected, sliding the sliding window in a designated area, and forming the training sample by using a first sample channel and a second sample channel; when a test sample is selected, sliding a sliding window on the whole graph, and forming the test sample by using a sample first channel and a sample second channel;
2.5 Randomly selecting 4% of the training samples as labeled training samples, and the rest as unlabeled training samples.
And 3, constructing a training network model.
3.1 Set SAR change detection dual network ψ 1 And psi is 2 ：
The two detection networks are composed of six layers of structures, wherein the first four layers are sharing layers, and the second two layers are non-sharing layers, and the two layers are:
3.11)Ψ 1 and psi is 2 Parameter setting and relation of each layer of the first four layers:
first full-connection layer L 1 The device is provided with 1000 neurons for extracting shallow features of a training sample and a test sample, and the shallow features of the training sample and the test sample generate an output vector of 1000 dimensions;
the second layer activation function layer ReLU is used for carrying out nonlinear mapping on the output of the upper layer full-connection layer, and the nonlinear mapping formula is as follows:
ReLU(x)＝max(0,x)，
wherein x is input ReLU (x) as output, and the dimension of the input and the output of the layer are consistent;
the third layer is a full connection layer L 2 1000 neurons are arranged for extracting deeper features from shallow features output by the upper full-connection layer, and the layer generates an output vector of 1000 dimensions;
the fourth layer is an activation function layer ReLU, and the action and principle of the fourth layer are consistent with those of the ReLU layer;
3.12)Ψ 1 fifth and sixth layer parameter settings and relationships:
Ψ 1 the fifth layer of (2) is a full connection layer L 31 The device is used for extracting different types of features from the output of the last active layer, wherein the input dimension of the layer is 1000, and the output dimension is 2;
Ψ 1 the sixth layer is Softmax classifier layer S 11 The function of this layer is to give L 31 The output 2-dimensional column vectors are respectively converted into classification probabilities, namely the probability that the current input sample belongs to a change class and a non-change class, and the samples are classified according to the probability values;
3.13)Ψ 2 fifth and sixth layer parameter settings and relationships:
Ψ 2 the fifth layer of (2) is a full connection layer L 32 The device is used for extracting different types of features from the output of the last active layer, wherein the input dimension of the layer is 1000, and the output dimension is 2;
Ψ 2 is a Softmax classifier layer S 21 The function of this layer is to give L 32 The 2-dimensional column vectors output by the layers are respectively converted into classification probabilities, namely the probability that the current input sample belongs to the change and non-change classes, and the samples are classified according to the probability values.
3.2 Setting up two discrimination networks
Figure BDA0002098559190000051
And->
Figure BDA0002098559190000052
The two discrimination networks are identical and are composed of six layers of structures, namely a first layer is a full-connection layer, a second layer is an activation function layer ReLU, a third layer is a full-connection layer, a fourth layer is an activation function layer, a fifth layer is a full-connection layer, a sixth layer is a Softmax classifier layer, and the parameter settings and relationships of the layers are as follows:
a first fully connected layer provided with 1000 neurons for extracting shallow discriminating features from the inputs of the network, this layer yielding an output vector of 1000 dimensions;
the second layer activates the function layer ReLU, is used for carrying on the nonlinear mapping to the output of the upper layer of all link layers, the dimension of input and output of this layer is unanimous;
the third layer is a full-connection layer, which is provided with 1000 neurons and is used for extracting deeper discrimination features from shallow features output by the upper full-connection layer, and the layer generates 1000-dimensional output vectors;
the fourth layer is an activation function layer ReLU;
the fifth layer is a full-connection layer and comprises 2 neurons, and is used for reducing the output dimension of the upper layer to a 2-dimensional vector for the subsequent classification probability calculation;
the sixth layer is a Softmax classifier layer, and the function of the layer is to convert the 2-dimensional column vector output by the previous layer into two-dimensional classification probability, namely the probability that the current input sample belongs to a true distribution sample and belongs to a generator generated sample, and judge the sample according to the probability value:
3.3 Connecting the dual network constructed in 3.1) with the two discrimination networks constructed in 3.2), namely the first discrimination network
Figure BDA0002098559190000053
Connected to a first detection network ψ 1 After the sixth layer, the second discrimination network +.>
Figure BDA0002098559190000054
Connected to a second detection network ψ 2 After the fifth layer, a training network model is formed.
And step 4, inputting training sample data into the training network model constructed in the step 3, and carrying out supervision training of the labeled data, countermeasure training of the unlabeled data and collaborative training in sequence to obtain a trained change detection network ψ.
4.1 Supervised training of tagged data):
4.11 For sending tagged data to the detection dual network ψ 1 、Ψ 2 In which a detection network ψ is calculated from labels fed into samples 1 Is a loss function L of (2) s1 And detecting network ψ 2 Is a loss function L of (2) s2 ：
Figure BDA0002098559190000055
Figure BDA0002098559190000056
Wherein v is 1 、v 2 For network ψ 1 Two values, z, of the two-dimensional vector of the output of the last fully connected layer 1 、z 2 For network ψ 2 I is the correct class label corresponding to the current input sample, i is 1 and indicates that the input sample is of a changed class, i is 2 and indicates that the input sample is of an unchanged class;
4.12 Using 4.11) the calculated first detection network ψ 1 Is a loss function L of (2) s1 And a second detection network ψ 2 Is a loss function L of (2) s2 Updating the first detection network ψ according to the gradient descent algorithm back propagation 1 Parameter and second detection network ψ 2 Parameters;
4.2 Performing countermeasure training on the unlabeled data;
4.2.1 To the first detection network ψ 1 As a generator network P 1 And with the first arbiter network
Figure BDA0002098559190000069
Together form a first generation-opposing network GAN 1 The label-free data is subjected to countermeasure training, and the following is realized:
firstly, a real sample and a generated sample are sequentially sent into a first judgmentNetwork of devices
Figure BDA0002098559190000062
In the above, the first discriminant network loss function is calculated according to the label of the input sample>
Figure BDA0002098559190000063
Figure BDA0002098559190000064
Wherein O is r1 And O f1 Two values of a two-dimensional vector output by the last full-connection layer of the discrimination network are respectively and correspondingly judged to be a real sample and a characteristic value of the generated sample, y 1 For network P 1 An output of (2);
then, calculating the gradient of the loss function of the first discriminant network, and updating the first discriminant network by back propagation of the gradient descent algorithm
Figure BDA0002098559190000065
Parameters;
then, the first generator network ψ 1 Generating a sample and sending the sample to a first discrimination network
Figure BDA0002098559190000066
Among them, calculate the first generator network P 1 Loss function->
Figure BDA0002098559190000067
Figure BDA0002098559190000068
Wherein O is r1 And O f1 Two values of a two-dimensional vector output by the last full-connection layer of the first judging network are respectively and correspondingly judged to be a real sample and a characteristic value judged to be a sample, y 1 For network P 1 An output of (2);
then, the gradient of the first generator network loss function is calculated, and the gradient descent algorithm is utilized to back-propagate and update the first generator network P 1 Parameters;
4.2.2 To the second detection network ψ 2 The first five layers of (a) are regarded as a generator network P 2 And with a second arbiter network
Figure BDA0002098559190000071
Together form a second generation countermeasure network GAN 2 Performing countermeasure training:
the first step, the real sample and the generated sample are sequentially sent into a second discriminator network, and a second discriminating network loss function is calculated according to the label sent into the sample
Figure BDA0002098559190000072
Figure BDA0002098559190000073
Wherein O is r2 And O f2 Two values of a two-dimensional vector output by the last full-connection layer of the second judging network are respectively and correspondingly judged to be a real sample and a characteristic value of the generated sample, and y 2 For network P 2 An output of (2);
step two, calculating the gradient of the loss function of the second discrimination network, and updating the network by using the back propagation of the loss
Figure BDA0002098559190000074
Parameters of (2);
third, the second generator network generates a sample and sends the sample to a second discrimination network
Figure BDA0002098559190000075
Among them, the second generator network P is calculated 2 Loss function->
Figure BDA0002098559190000076
Figure BDA0002098559190000077
Wherein O is r2 And O f2 Two values of a two-dimensional vector output by the last full-connection layer of the second judging network are respectively and correspondingly judged to be a real sample and a characteristic value of the generated sample, and y 2 For the second generator network P 2 An output of (2);
fourth, calculating the gradient of the loss function of the second generator network, and back-propagating and updating the second generator network P by using a gradient descent algorithm 2 Parameters.
4.3 Respectively and simultaneously sending the unlabeled data into the detection dual network ψ 1 And psi is 2 The following co-training was performed:
4.31 First detection network ψ 1 Extracting category characteristics from the unlabeled data, and classifying the category characteristics by using a Softmax classifier to obtain pseudo-label PL corresponding to each unlabeled sample 1 And a classification probability vector py, calculating the confidence of the ith unlabeled exemplar classification as: cony i ＝max(py i ) And confidence of the classification Cony i With a preset confidence threshold T y Comparing the sample with confidence greater than the threshold with the labeled sample to obtain a second detection network ψ 2 Is a supervised training sample;
4.32 Second detection network ψ 2 Extracting mode features from the label-free data, and classifying the mode features by using a Softmax classifier to obtain pseudo labels PL corresponding to each label-free sample 2 And classifying the probability vector pz, and calculating the confidence of the ith label-free sample classification as follows: conz i ＝max(pz i ) And will Conz i With a preset confidence threshold T z Comparing the samples with confidence greater than the threshold with the labeled samples to obtain a first detection network ψ 1 Is a training sample of supervised training;
the trained change detection network ψ is obtained by iterating the above step 4.1) supervised training of the tagged data, 4.2) countermeasure training of the untagged data and 4.3) co-training of the untagged data.
And 5, inputting the test sample data into a trained change detection network ψ for detection to obtain a change detection result of the SAR image.
The effect of the invention can be further illustrated by the following experimental data:
experimental conditions
1) Experimental data
This experiment uses four sets of SAR image data, as shown in fig. 3, in which:
fig. 3 (a 1) and 3 (a 2) are images obtained by ERS-2SAR sensors at 8/5/2003 and 256 x 256 in image size, respectively, and fig. 3 (a 3) is a corresponding change detection reference map, this group being called San Francisco data.
Fig. 3 (b 1) and 3 (b 2) are SAR images obtained by a Radarsat-1 sensor at 7, 1997 and 8, respectively, with an image size of 290 x 350, and fig. 3 (b 3) is a corresponding change detection reference map, this group being called Ottawa data.
Fig. 3 (c 1) and 3 (c 2) are SAR images obtained by a Radarsat-2 sensor at month 6 of 2008 and month 6 of 2009, respectively, the image size is 289×257, and fig. 3 (c 3) is a corresponding change detection reference map, which is referred to as Yellow River Farmland I data in this group.
Fig. 3 (d 1) and 3 (d 2) are SAR images obtained by a Radarsat-2 sensor at month 6 of 2008 and month 6 of 2009, respectively, the image size is 291×306, and fig. 3 (d 3) is a corresponding change detection reference map, which is referred to as Yellow River Farmland II data in this group.
2) Evaluation criterion
The experimental results were evaluated using the following criteria
False alarm rate FA, false alarm rate MD, overall error rate OE, classification accuracy PCC and Kappa coefficient KC.
Second, experimental details
Experiment one: the invention and the deep neural network DNN supervision algorithm A1 are used for carrying out a change detection comparison experiment on the data against the self-encoder SAAE semi-supervision algorithm A2 and the semi-supervision deep neural network SSDC semi-supervision algorithm A3 combined with cooperative training, and the comparison result of the performance parameters is shown in the table 1.
TABLE 1 comparison of the performance parameters of the inventive method and the related models
Figure BDA0002098559190000091
In table 1: semi-supervised deep neural network SSDC semi-supervised algorithm A3 experiments compared to the lack of the countermeasure training section of the present invention, the deep neural network DNN supervised algorithm A1 experiments used the same network as the present invention, but only the supervised training process.
As can be seen from Table 1, the best results were obtained with the present invention, which also performed more stably than the other methods. The experimental result of SSDC semi-supervised algorithm of the semi-supervised deep neural network is compared, so that the increase of the resistance training is reasonable, and the classification effect is improved.
Experiment II: the invention combines with the existing PCA and K-means combined unsupervised change detection algorithm PCAKM, gabor transformation and two-stage clustering combined unsupervised change detection method GaborTLC, PCANet based unsupervised change detection method PCANet and ELM based unsupervised change detection method of overrun learning machine, and the data are subjected to change detection comparison experiments, and the performance parameter comparison results are shown in Table 2.
TABLE 2 comparison of performance parameters of the inventive method with the existing unsupervised method
Figure BDA0002098559190000101
As can be seen from Table 2, the invention has better performance, because the semi-supervised model can extract the identification information from the labeled sample and the unlabeled sample, and the collaborative training algorithm improves the generalization performance by introducing the pseudo-label training sample, so the invention obtains better detection result than the existing method.
Experiment III: the result of the change detection comparison experiment of the data obtained by the method of the present invention with the prior art method used in experiment one and experiment two is shown in fig. 4, wherein:
FIG. 4 (a 1) is a graph showing the results of the A2 method on San Francisco data;
FIG. 4 (a 2) is a graph showing the results of the A1 method on San Francisco data;
FIG. 4 (A3) is a graph showing the results of the A3 method on San Francisco data;
FIG. 4 (a 4) is a graph showing the results of the detection of the present invention on San Francisco data;
FIG. 4 (a 5) is a graph of the real variation area of the San Francisco data;
FIG. 4 (a 6) is a graph showing the results of the PCAKM method on San Francisco data;
FIG. 4 (a 7) is a graph showing the detection results of GaborTLC on San Francisco data;
FIG. 4 (a 8) is a graph showing the results of the PCANet method on San Francisco data;
FIG. 4 (a 9) is a graph showing the detection results of the ELM method on San Francisco data;
FIG. 4 (b 1) is a graph showing the results of the A2 method on Ottawa data;
FIG. 4 (b 2) is a graph showing the results of the A1 method on Ottawa data;
FIG. 4 (b 3) is a graph showing the results of the A3 method on Ottawa data;
FIG. 4 (b 4) is a graph showing the detection results of the present invention on Ottawa data;
FIG. 4 (b 5) is a graph of the true change area of Ottawa data;
FIG. 4 (b 6) is a graph showing the detection result of the PCAKM method on Ottawa data;
FIG. 4 (b 7) is a graph showing the detection results of GaborTLC method on Ottawa data;
FIG. 4 (b 8) is a graph showing the detection result of the PCANet method on Ottawa data;
FIG. 4 (b 9) is a graph showing the detection result of the ELM method on Ottawa data;
FIG. 4 (c 1) is a graph showing the detection result of the A2 method on Yellow River Farmland I data;
FIG. 4 (c 2) is a graph showing the detection result of the A1 method on Yellow River Farmland I data;
FIG. 4 (c 3) is a graph showing the detection result of the A3 method on Yellow River Farmland I data;
FIG. 4 (c 4) is a graph showing the detection result of Yellow River Farmland I data according to the present invention;
FIG. 4 (c 5) is a graph of the true variation area of Yellow River Farmland I data;
FIG. 4 (c 6) is a graph showing the detection result of the PCAKM method on Yellow River Farmland I data;
FIG. 4 (c 7) is a graph showing the detection results of GaborTLC method on Yellow River Farmland I data;
FIG. 4 (c 8) is a graph showing the detection result of the PCANet method on Yellow River Farmland I data;
FIG. 4 (c 9) is a graph showing the detection results of the ELM method on Yellow River Farmland I data;
FIG. 4 (d 1) is a graph showing the detection result of the A2 method on Yellow River Farmland II data;
FIG. 4 (d 2) is a graph showing the detection result of the A1 method on Yellow River Farmland II data;
FIG. 4 (d 3) is a graph showing the detection result of the A3 method on Yellow River Farmland II data;
FIG. 4 (d 4) is a graph showing the detection result of Yellow River Farmland II data according to the present invention;
FIG. 4 (d 5) is a graph of the true variation area of Yellow River Farmland II data;
FIG. 4 (d 6) is a graph showing the detection result of the PCAKM method on Yellow River Farmland II data;
FIG. 4 (d 7) is a graph showing the detection results of GaborTLC method on Yellow River Farmland II data;
FIG. 4 (d 8) is a graph showing the detection result of the PCANet method on Yellow River Farmland II data;
FIG. 4 (d 9) is a graph showing the detection results of the ELM method on Yellow River Farmland II data.
As can be seen from fig. 4, the detection result graph of the invention is closer to the real change area graph, can more accurately reflect the shape of the change area, and has better detection effect.
The above description is only one specific example of the invention and does not constitute any limitation of the invention, and it will be apparent to those skilled in the art that various modifications and changes in form and details may be made without departing from the principles, construction of the invention, but these modifications and changes based on the idea of the invention are still within the scope of the claims of the invention.

Claims (5)

1. The SAR image change detection method based on the semi-supervised countermeasure depth network is characterized by comprising the following steps of:
(1) Calculating a logarithmic ratio difference graph K of the two images by using the two-phase SAR image data;
(2) Extracting training samples and test samples on the two-time-phase SAR image and the difference image in a sliding window mode, randomly selecting 4% of the training samples as labeled training samples, and taking the rest as unlabeled training samples;
(3) Building a training network model:
(3a) Setting SAR change detection dual network ψ 1 And psi is 2 ：
Each network is composed of six layers, wherein the first four layers are sharing layers, namely the first layer is a full-connection layer L 1 The second layer is an activation function layer ReLU and the third layer is a full connection layer L 2 The fourth layer is an activation function layer ReLU, and the fifth and sixth are non-shared layers, wherein:
first network ψ 1 The fifth layer of (2) is a full connection layer L 13 The sixth layer is Softmax classifier layer S 11 ，
Second network ψ 2 The fifth layer of (2) is a full connection layer L 23 The sixth layer is Softmax classifier layer S 21 ；
(3b) Setting two discrimination networks
Figure QLYQS_1
And->
Figure QLYQS_2
The two discrimination networks are identical and are composed of six layers of structures, namely a first layer is a full-connection layer, a second layer is an activation function layer ReLU, a third layer is a full-connection layer, a fourth layer is an activation function layer, a fifth layer is a full-connection layer, and a sixth layer is a Softmax classifier layer;
(3c) Connecting the dual network with two discrimination networks, i.e. the first discrimination network
Figure QLYQS_3
Connected to a first detection network ψ 1 After that, the second discrimination network->
Figure QLYQS_4
Connected to a second detection network ψ 2 Then, forming a training network model;
(4) Inputting training sample data into the constructed training network model (3), and carrying out supervision training of the labeled data, countermeasure training of the unlabeled data and collaborative training in sequence in an iterated manner to obtain a trained change detection network ψ;
the supervision training of the tagged data is to send the tagged data to the detection dual network ψ 1 、Ψ 2 The used loss function is a two-class cross entropy loss function, and the formula is as follows:
Figure QLYQS_5
Figure QLYQS_6
wherein v is 1 、v 2 For network ψ 1 Two values, z, of the two-dimensional vector of the output of the last fully connected layer 1 、z 2 For network ψ 2 I is the correct class label corresponding to the current input sample, i is 1 and indicates that the input sample is of a changed class, i is 2 and indicates that the input sample is of an unchanged class;
the challenge training for unlabeled data is achieved as follows:
(4a) Will first detect network ψ 1 As a generator network P 1 And with the first arbiter network
Figure QLYQS_7
Together form a first generation-opposing network GAN 1 Challenge training was performed on unlabeled data:
4a1) Sequentially feeding the real sample and the generated sample into a first discriminator network
Figure QLYQS_8
In the above, the first discriminant network loss function is calculated according to the label of the input sample>
Figure QLYQS_9
Figure QLYQS_10
/>
Wherein O is r1 And O f1 Two values of a two-dimensional vector output by the last full-connection layer of the first judging network are respectively and correspondingly judged to be a real sample and a characteristic value judged to be a sample, y 1 For network P 1 An output of (2);
4a2) Calculating the gradient of the loss function of the first discriminant network, and updating the first discriminant network by using the backward propagation of the gradient descent algorithm
Figure QLYQS_11
Parameters;
4a3) Will first generator network P 1 Generating a sample and sending the sample to a first discrimination network
Figure QLYQS_12
Among them, calculate the first generator network P 1 Loss function->
Figure QLYQS_13
Figure QLYQS_14
Wherein O is r1 And O f1 Two values of a two-dimensional vector output by the last full-connection layer of the first judging network are respectively and correspondingly judged to be a real sample and a characteristic value judged to be a sample, y 1 For network P 1 An output of (2);
4a4) Calculating the gradient of the loss function of the first generator network, and back-propagating and updating the first generator network P by using a gradient descent algorithm 1 Parameters;
(4b) Will second detection network ψ 2 The first five layers of (a) are regarded as a generator network P 2 And with a second arbiter network
Figure QLYQS_15
Together form a second generation countermeasure network GAN 2 Performing countermeasure training:
4b1) Sequentially sending the real sample and the generated sample into a second discriminator network, and calculating a second discrimination network loss function according to the label of the sent sample
Figure QLYQS_16
Figure QLYQS_17
Wherein O is r2 And O f2 Two values of a two-dimensional vector output by the last full-connection layer of the second judging network are respectively and correspondingly judged to be a real sample and a characteristic value of the generated sample, and y 2 For network P 2 An output of (2);
4b2) Calculating a second discriminant network loss function gradient, updating the network with back propagation of losses
Figure QLYQS_18
Parameters of (2);
4b3) Sending the second generator network generated sample to a second discrimination network
Figure QLYQS_19
Among them, the second generator network P is calculated 2 Loss function->
Figure QLYQS_20
Figure QLYQS_21
Wherein O is r2 And O f2 Two values of a two-dimensional vector output by the last full-connection layer of the second judging network are respectively and correspondingly judged to be a real sample and a characteristic value of the generated sample, and y 2 For the second generator network P 2 An output of (2);
4b4) Calculating a second generator network loss function gradient, and back-propagating and updating the second generator network P by using a gradient descent algorithm 2 Parameters;
the co-training of the non-label data is to send the non-label data into the detection dual-network ψ at the same time 1 And psi is 2 The following co-training was performed:
4c) First detection network ψ 1 Extracting category characteristics from the unlabeled data, and classifying the category characteristics by using a Softmax classifier to obtain pseudo-label PL corresponding to each unlabeled sample 1 And a classification probability vector py, calculating the confidence of the ith unlabeled exemplar classification as: cony i ＝max(py i ) And confidence of the classification Cony i With a preset confidence threshold T y Comparing, using the sample with confidence greater than the threshold as the second detection network ψ 2 Is a supervised training sample;
4d) Second detection network ψ 2 Extracting mode features from the label-free data, and classifying the mode features by using a Softmax classifier to obtain pseudo labels PL corresponding to each label-free sample 2 The classification probability vector pz,the confidence of the ith unlabeled exemplar class is calculated as: conz i ＝max(pz i ) And will Conz i With a preset confidence threshold T z Comparing, using a sample with confidence greater than a threshold as the first detection network ψ 1 Is a training sample of supervised training;
(5) And inputting the test sample data into a trained change detection network ψ for detection to obtain a change detection result of the SAR image.
2. The method of claim 1, wherein the corresponding log-ratio disparity map is calculated in step (1) using the two-phase SAR image data, as follows:
Figure QLYQS_22
wherein I is 1 For the first time phase diagram of the original SAR image, I 2 Is the second time-phase diagram of the original SAR image.
3. The method of claim 1, wherein the training samples and the test samples are selected on the two-phase SAR image and the disparity map using a sliding window model in (2) as follows:
2a) Setting the size of a sliding window as N multiplied by N, setting the center of the sliding window as (i, j), for a training sample, carrying out sliding window selection image blocks in a designated area, and for a test sample, carrying out sliding window selection image blocks on the whole image;
2b) At two time phase image I 1 And I 2 In (i, j), image blocks of size N×N are selected as the center
Figure QLYQS_23
And->
Figure QLYQS_24
Will->
Figure QLYQS_25
And->
Figure QLYQS_26
Splicing together along a first dimension to obtain a sample first channel with the size of 2NxN;
2c) Corresponding to two-phase image I 1 And I 2 A middle pixel point (i, j), and selecting an image block centered on (i, j) in the difference image K as a mark
Figure QLYQS_27
The size of the image block is N, calculating +.>
Figure QLYQS_28
Extending m into a matrix of size 2N x N as a sample second channel;
2d) When a training sample is selected, the sliding window slides in a designated area, the training sample is formed by the first sample channel and the second sample channel, and when a test sample is selected, the sliding window slides in the whole graph, and the test sample is formed by the first sample channel and the second sample channel.
4. The method of claim 1, wherein the change in (3 a) detects a dual network structure ψ 1 And psi is 2 The parameter settings and relation of each layer are as follows:
Ψ 1 and psi is 2 The first four layers of (a) are shared layers, wherein:
first full-connection layer L 1 The device is provided with 1000 neurons for extracting shallow features of a training sample and a test sample, and the shallow features of the training sample and the test sample generate an output vector of 1000 dimensions;
the second layer activation function layer ReLU is used for carrying out nonlinear mapping on the output of the upper layer full-connection layer, and the nonlinear mapping formula is as follows:
ReLU(x)＝max(0,x)
wherein x is input ReLU (x) is output, and the dimension of the input and the output of the layer are consistent;
the third layer is a full connection layer L 2 Which is provided with 1000 neurons for extracting deeper features from shallow features output from the upper full-connection layer,this layer produces a 1000-dimensional output vector;
the fourth layer is an activation function layer ReLU, and the action and principle of the fourth layer are consistent with those of the ReLU layer;
Ψ 1 and psi is 2 Is a non-shared layer, wherein:
Ψ 1 the fifth layer of (2) is a full connection layer L 31 The device is used for extracting different types of features from the output of the last active layer, wherein the input dimension of the layer is 1000, and the output dimension is 2;
Ψ 1 the sixth layer is Softmax classifier layer S 11 The function of this layer is to give L 31 The output 2-dimensional column vectors are respectively converted into classification probabilities, namely the probability that the current input sample belongs to a change class and a non-change class, and the samples are classified according to the probability values;
Ψ 2 the fifth layer is a full connection layer L 32 The device is used for extracting different types of features from the output of the last active layer, wherein the input dimension of the layer is 1000, and the output dimension is 2;
Ψ 2 the sixth layer is Softmax classifier layer S 21 The function of this layer is to give L 32 The 2-dimensional column vectors output by the layers are respectively converted into classification probabilities, namely the probability that the current input sample belongs to the change and non-change classes, and the samples are classified according to the probability values.
5. The method of claim 1, wherein two of (3 b) discriminate networks
Figure QLYQS_29
And->
Figure QLYQS_30
The structure is the same, and the parameter settings and the relation of each layer are as follows:
a first fully connected layer provided with 1000 neurons for extracting shallow discriminating features from the inputs of the network, this layer yielding an output vector of 1000 dimensions;
the second layer activates the function layer ReLU, is used for carrying on the nonlinear mapping to the output of the upper layer of all link layers, the dimension of input and output of this layer is unanimous;
the third layer is a full-connection layer, which is provided with 1000 neurons and is used for extracting deeper discrimination features from shallow features output by the upper full-connection layer, and the layer generates 1000-dimensional output vectors;
the fourth layer is an activation function layer ReLU;
the fifth layer is a full-connection layer and comprises 2 neurons, and is used for reducing the output dimension of the upper layer to a 2-dimensional vector for the subsequent classification probability calculation;
the sixth layer is a Softmax classifier layer, and the function of the layer is to convert the 2-dimensional column vector output by the previous layer into two-dimensional classification probability, namely the probability that the current input sample belongs to a true distribution sample and belongs to a generator generated sample, and judge the sample according to the probability value.
CN201910527007.1A 2019-06-18 2019-06-18 SAR image change detection method based on semi-supervised countermeasure depth network Active CN110263845B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910527007.1A CN110263845B (en) 2019-06-18 2019-06-18 SAR image change detection method based on semi-supervised countermeasure depth network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910527007.1A CN110263845B (en) 2019-06-18 2019-06-18 SAR image change detection method based on semi-supervised countermeasure depth network

Publications (2)

Publication Number Publication Date
CN110263845A CN110263845A (en) 2019-09-20
CN110263845B true CN110263845B (en) 2023-05-02

Family

ID=67919058

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910527007.1A Active CN110263845B (en) 2019-06-18 2019-06-18 SAR image change detection method based on semi-supervised countermeasure depth network

Country Status (1)

Country Link
CN (1) CN110263845B (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111444955B (en) * 2020-03-25 2022-08-02 哈尔滨工程大学 Underwater sonar image unsupervised classification method based on class consciousness field self-adaption
CN111523422B (en) * 2020-04-15 2023-10-10 北京华捷艾米科技有限公司 Key point detection model training method, key point detection method and device
CN112257855B (en) * 2020-11-26 2022-08-16 Oppo（重庆）智能科技有限公司 Neural network training method and device, electronic equipment and storage medium
CN112285664B (en) * 2020-12-18 2021-04-06 南京信息工程大学 Method for evaluating countermeasure simulation confidence of radar-aircraft system
CN112686305A (en) * 2020-12-29 2021-04-20 深圳龙岗智能视听研究院 Semi-supervised learning method and system under assistance of self-supervised learning
CN112766381B (en) * 2021-01-22 2023-01-24 西安电子科技大学 Attribute-guided SAR image generation method under limited sample
CN112784777B (en) * 2021-01-28 2023-06-02 西安电子科技大学 Unsupervised hyperspectral image change detection method based on countermeasure learning
CN113255451B (en) * 2021-04-25 2023-04-07 西北工业大学 Method and device for detecting change of remote sensing image, electronic equipment and storage medium
CN114301637B (en) * 2021-12-11 2022-09-02 河南大学 Intrusion detection method and system for medical Internet of things
CN114821299B (en) * 2022-03-28 2024-03-19 西北工业大学 Remote sensing image change detection method
CN114821337B (en) * 2022-05-20 2024-04-16 武汉大学 Semi-supervised SAR image building area extraction method based on phase consistency pseudo tag
CN116127345B (en) * 2022-12-23 2023-11-14 北京科技大学 Converter steelmaking process mode design method based on deep clustering generation countermeasure network

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3246875A2 (en) * 2016-05-18 2017-11-22 Siemens Healthcare GmbH Method and system for image registration using an intelligent artificial agent
CN107563428A (en) * 2017-08-25 2018-01-09 西安电子科技大学 Classification of Polarimetric SAR Image method based on generation confrontation network
CN107977667A (en) * 2016-10-21 2018-05-01 西安电子科技大学 SAR target discrimination methods based on semi-supervised coorinated training
CN108492298A (en) * 2018-04-13 2018-09-04 西安电子科技大学 Based on the multispectral image change detecting method for generating confrontation network
CN108564115A (en) * 2018-03-30 2018-09-21 西安电子科技大学 Semi-supervised polarization SAR terrain classification method based on full convolution GAN
CN109242889A (en) * 2018-08-27 2019-01-18 大连理工大学 SAR image change detection based on context conspicuousness detection and SAE

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3246875A2 (en) * 2016-05-18 2017-11-22 Siemens Healthcare GmbH Method and system for image registration using an intelligent artificial agent
CN107977667A (en) * 2016-10-21 2018-05-01 西安电子科技大学 SAR target discrimination methods based on semi-supervised coorinated training
CN107563428A (en) * 2017-08-25 2018-01-09 西安电子科技大学 Classification of Polarimetric SAR Image method based on generation confrontation network
CN108564115A (en) * 2018-03-30 2018-09-21 西安电子科技大学 Semi-supervised polarization SAR terrain classification method based on full convolution GAN
CN108492298A (en) * 2018-04-13 2018-09-04 西安电子科技大学 Based on the multispectral image change detecting method for generating confrontation network
CN109242889A (en) * 2018-08-27 2019-01-18 大连理工大学 SAR image change detection based on context conspicuousness detection and SAE

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
《A semi-supervised generative framework with deep learning features for high-resolution remote sensing image scene classification》;Han W等;《ISPRS Journal of Photogrammetry and Remote Sensing》;20181231;全文 *
《基于卷积神经网络的SAR图像目标检测算法》;杜兰等;《电子与信息学报》;20161231;全文 *
《小样本下的极化SAR图像分类问题研究》;滑文强;《中国博士学位论文全文数据库》;20190215(第02期);全文 *

Also Published As

Publication number Publication date
CN110263845A (en) 2019-09-20

Similar Documents

Publication Publication Date Title
CN110263845B (en) SAR image change detection method based on semi-supervised countermeasure depth network
CN109800648B (en) Face detection and recognition method and device based on face key point correction
Jia et al. A semisupervised Siamese network for hyperspectral image classification
CN107330396B (en) Pedestrian re-identification method based on multi-attribute and multi-strategy fusion learning
Zhou et al. Robust matching for SAR and optical images using multiscale convolutional gradient features
CN110443143B (en) Multi-branch convolutional neural network fused remote sensing image scene classification method
CN106096561B (en) Infrared pedestrian detection method based on image block deep learning features
CN108830296B (en) Improved high-resolution remote sensing image classification method based on deep learning
Liu et al. Visual loop closure detection with a compact image descriptor
CN108960140B (en) Pedestrian re-identification method based on multi-region feature extraction and fusion
WO2019232894A1 (en) Complex scene-based human body key point detection system and method
CN111126360A (en) Cross-domain pedestrian re-identification method based on unsupervised combined multi-loss model
CN108492298B (en) Multispectral image change detection method based on generation countermeasure network
CN105528794A (en) Moving object detection method based on Gaussian mixture model and superpixel segmentation
CN111709311A (en) Pedestrian re-identification method based on multi-scale convolution feature fusion
CN106844739B (en) Remote sensing image change information retrieval method based on neural network collaborative training
CN109801305B (en) SAR image change detection method based on deep capsule network
CN109902585A (en) A kind of three modality fusion recognition methods of finger based on graph model
CN114821198B (en) Cross-domain hyperspectral image classification method based on self-supervision and small sample learning
CN115527269B (en) Intelligent human body posture image recognition method and system
Yang et al. Dynamic fractal texture analysis for PolSAR land cover classification
Su et al. FSRDD: An efficient few-shot detector for rare city road damage detection
CN115131580A (en) Space target small sample identification method based on attention mechanism
CN113808166B (en) Single-target tracking method based on clustering difference and depth twin convolutional neural network
Li et al. Detecting building changes using multi-modal Siamese multi-task networks from very high resolution satellite images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant