CN115761502A - SAR Image Change Detection Method Based on Hybrid Convolution - Google Patents

Abstract

The invention discloses a SAR image change detection method based on mixed convolution, which mainly solves the problems that the prior art is difficult to extract the global information of an image and lacks a reliable label training network under the condition of no label, and the scheme is as follows: constructing three-channel input and initial labels; building a graph convolution enhanced convolutional network (GECN) formed by cascading a feature extraction module, a progressive fusion module and a label updating module; training the GECN by using three-channel input and labels; and inputting the tested SAR image into the trained GECN to obtain a change detection result. According to the invention, the multi-scale features of the local and global information of the SAR image are obtained through the feature extraction module, so that the feature extraction capability of the GECN network is improved; more reliable labels are obtained through the label updating module to train the GECN, the generalization performance of the GECN is improved, and the method can be used for urban planning layout, natural disaster assessment and military dynamic reconnaissance.

Description

SAR Image Change Detection Method Based on Hybrid Convolution

technical field

The invention belongs to the technical field of image processing, and relates to a method for detecting changes in synthetic aperture radar (SAR) images, which can be used for urban planning layout, natural disaster assessment and military dynamic reconnaissance.

Background technique

Change detection in SAR images plays a vital role in the field of remote sensing and has received increasing attention due to its wide range of applications. Existing SAR image change detection techniques are mainly divided into two categories.

The first category is a change detection method based on a traditional algorithm, which takes the feature difference and ratio of a pixel pair or an object pair as input in the region recognition stage, and detects changes through a threshold. According to the size of the image processing unit, it can be further divided into object-based methods and pixel-based methods. However, both pixel-based and object-based change detection methods only use manually extracted features, such as spectral features, texture features, and shape features to extract change information, because these features cannot fully represent the key information of SAR images. It greatly affects the change detection results.

The second category is change detection based on deep learning algorithm, which uses the powerful nonlinear fitting ability and multi-level structure of convolutional neural network, so that the learned features have advanced semantic information and rich spatial context information, so It is a widely used change detection method. Although the change detection method based on deep learning has achieved great success in the change detection task of SAR images, there are still some problems to be further solved. First of all, only a multi-level convolutional neural network is used to extract the features of SAR images. In theory, deep features have global context information, but in fact, the receptive field of deep features is not global, and at the same time, shallow features cannot be effectively used. Global information, thus lacking global information to model image features. Secondly, the general fusion of SAR multi-scale features is to use up-down sampling to the same size, and then directly add or perform channel splicing. However, in practice, only one-time processing cannot fuse multi-scale features well, and the features will be biased to a certain level. One-scale features do not make full use of multi-scale features. Finally, most unsupervised change detection algorithms use clustering to obtain training pseudo-labels, so the selected training labels are mixed with a large number of wrong labels, and their reliability is low, so that the network training cannot achieve the expected results.

Sun Yuli proposed a graph based on nonlocal patch similarity (the nonlocal patch similarity based graph, NPSG) to measure the structural consistency between heterogeneous images, so as to complete the change detection task. The specific implementation is as follows: first, the image is divided into a series of patches, and in forward detection, for each target patch in the pre-event image, its k-nearest NPSG, and then map this with the nearest NPSG of the post-event image, and compare the nearest NPSG of its own post-event image with the nearest NPSG of the pre-event image by calculating the similarity difference; method to get the change detection result. This method uses the k-nearest neighbor graph structure to model the global context of the target patch. Although it can achieve better change detection accuracy, it requires a lot of computing resources when calculating the similarity between different patches.

Dong Huihui and others proposed a SAR image change detection method based on multi-scale self-attention deep clustering MSDC in Institute of Electrical and Electronics Engineers IEEE, 2021, 60:1-16. It first uses the side window filter algorithm to reduce the speckle noise in the SAR image; then, for the processed image, the multi-scale adjacent area of the given central pixel is used as the basic difference analysis unit of the sample; then, the multi-scale features are analyzed Fusion, to obtain the difference features of the two branches, and finally, use clustering to obtain labels to optimize the network. Among them, in the process of multi-scale feature fusion, first use the convolution nonlinear transformation to adjust the features of the three scales to the same size, and then these three features of the same size are considered as the query q and key values in the self-attention module k. The value v uses the self-attention mechanism to obtain a feature; finally, the feature obtained by self-attention is added to the feature before self-attention to complete the fusion of multi-scale features. Dong Huihui's ablation experiment proved the effectiveness of multi-scale attention, but since the generation of the weight matrix in the self-attention mechanism requires a large amount of computing resources to support, the computational cost is too high and it is not easy to apply.

Qu Xiaofan and others published a change detection method for SAR images using the dual-domain network DDNet on IEEE, 2021, 19:1-5. It uses the feature representation of frequency domain and space domain to alleviate speckle noise, and uses hierarchical fuzzy aggregation Classes are labeled, and 10% of the reliable labels are selected according to the degree of membership to train the network. Although this method can effectively alleviate the number of wrong labels, the number of labels participating in the training is small, which hinders the further improvement of network performance.

Contents of the invention

The purpose of the present invention is to address the shortcomings of the above-mentioned prior art, and propose a SAR image change detection method based on hybrid convolution to extract multi-scale features with global information and local information, improve detection accuracy, and reduce computing resources.

For realizing the above object, the technical scheme of the present invention comprises the following steps:

(1) Build a graph convolution enhanced convolutional network GECN:

1a) Establish a feature extraction module including convolutional neural network submodule CNNB, image volume neural network submodule GCNB and fusion submodule, wherein CNNB and GCNB are connected in parallel and then connected in series with the fusion submodule;

1b) Establish a progressive fusion module that sequentially performs up-and-down sampling operations, splicing operations, and convolution operations;

1c) Establish a label update module that updates the training label by voting on the network prediction label and the prototype label;

1d) The feature extraction module, progressive fusion module and label update module are connected in series to form a graph convolution enhanced convolutional network GECN;

(2) Obtain the input and initial label of GECN:

2a) Use the logarithmic ratio operator to obtain the difference map for the dual-time SAR image, and stitch the difference image and the dual-time image in the channel dimension to obtain the input of the three-channel GECN;

2b) Clustering the difference map in 2a) using a fuzzy clustering algorithm to obtain an initial label;

(3) Use the input and label of GECN to train GECN:

3a) In the initial 20 training batches, the input of the three channels is sent to GECN, the output of GECN and the initial label are applied to the cross-entropy loss function, and the parameters of GECN are updated using the gradient descent method to complete the pre-training of GECN;

3b) In the next 50 training batches, the three-channel input is sent to the pre-trained GECN, the initial label is updated using the label update module, and cross-entropy is used for the output of the pre-trained GECN and the updated initial label Loss function, and use the gradient descent method to further update the pre-trained GECN parameters to complete the GECN training;

(4) Input the tested SAR image into the trained GECN to get the GECN output, use the max function on the GECN output to get the category of each pixel sample on the SAR image, which is the change detection result of the final SAR image.

Compared with the prior art, the present invention has the following advantages:

1) The present invention uses the point vector on the compressed feature as a graph node through the graph volume neural network sub-module GCNB, and performs graph convolution operations along the x-axis and y-axis respectively, so that images can be obtained with little information loss The global information of the image is not only easy to operate, but also can fully mine the global information of the image.

2) Since the present invention uses an adaptive fusion method to fuse the local and global information of the image, it can more fully explore useful information for the change detection task.

3) Because the present invention embeds the difference map into the input, it is equivalent to giving the prior information of GECN training, which makes GECN converge faster and reduces the occupancy of computing resources in the training process.

4) The present invention uses a step-by-step fusion method to fully explore the multi-scale information of the image, and avoids the problem that features are biased towards features of a certain scale.

5) The present invention uses the tag update module to update the initial tags, and obtains more and more reliable tags, and uses these tags to train GECN, which can increase the generalization of GECN and improve the change detection accuracy of GECN.

Simulation experiments show that the present invention can achieve excellent change detection performance.

Description of drawings

In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the specific embodiments or the prior art.

Fig. 1 is the realization flowchart of the present invention;

Fig. 2 is graph convolution strengthened convolutional network GECN frame diagram among the present invention;

Fig. 3 is the graph convolutional neural network submodule GCNB structural representation of GECN network among the present invention;

Fig. 4 is the structural representation of the fusion submodule of GECN network among the present invention;

Fig. 5 is a schematic diagram of a tag update module in a GECN network in the present invention;

Fig. 6 is a diagram of simulation results of SAR image change detection using the present invention and the existing detection method respectively.

Detailed ways

The examples and effects of the present invention will be described in further detail below in conjunction with the accompanying drawings.

Referring to Figure 1, in this example, the SAR image change detection method based on hybrid convolution includes obtaining three-channel input and initial labels, building a graph convolution enhanced convolutional network GECN, and using three-channel input and labels to train GECN and change the results There are four parts to obtain the graph, and the specific implementation is as follows:

Step 1: Obtain the input and initial labels of the three channels.

1.1) Construct three-channel input:

Assuming that there are two registered SAR images I ₁ and I ₂ acquired at the same location and at different times, in order to identify the area of change between I ₁ and I ₂ and reduce the impact of speckle noise in the SAR image on change detection performance, it first uses the logarithmic ratio operator to smooth the speckle noise on the images I ₁ and I ₂ to obtain the difference map D; in order to compensate for the information loss of D caused by the smoothing effect of the logarithmic ratio operator, I ₁ , I ₂ and the difference map D are concatenated in the channel dimension to obtain a three-channel input.

1.2) Get the initial label:

Use the fuzzy clustering algorithm to cluster the difference graph D in 1.1), and its working process is as follows:

1.2.1) Determine the number c of fuzzy clustering categories and the number of iterations t;

1.2.2) Initialize the membership matrix u with a random number between 0 and 1, so that the sum is 1:

Among them, u _ij is the membership degree of the j-th pixel sample belonging to the i-th category, and n is the number of pixel samples;

1.2.3) Calculate the cluster center according to the degree of membership:

Among them, _ci is the i-th cluster center, and x _j is the j-th pixel sample;

1.2.4) Calculate the sum of squared errors J from the pixel point sample to various center points:

1.2.5) Find the extreme value of 1.2.4) formula, when the error sum of squares is minimum, get the updated membership degree matrix u _ij ;

1.2.6) Repeat 1.2.3) to 1.2.5) until the number of iterations t is reached to obtain the initial label.

Step 2, build a graph convolution enhanced convolutional network GECN.

Referring to Figure 2, GECN includes a series-connected feature extraction module, a progressive fusion module, and a label update module. The building process of each module is as follows:

2.1) Establish a feature extraction module comprising the convolutional neural network submodule CNNB, the graph convolutional neural network submodule GCNB and the fusion submodule:

2.1.1) Three convolution layers with convolution kernel sizes of 1×1, 3×3, and 1×1 are sequentially connected to form a CNNB submodule, which is used to explore the local information of the image;

2.1.2) Establish GCNB submodule:

Referring to Figure 3, the GCNB sub-module includes two parallel pooling layers and four graph convolution layers. The four graph convolution layers are connected in series and connected to each pooling layer respectively. The working process of the GCNB sub-module as follows:

First, two parallel pooling layers perform global average pooling on the input F _in of GCNB along the x-axis direction and the y-axis direction respectively, and obtain compressed features F _x and F _y :

F _x = Pooling _x (F _in )

F _y = Pooling _y (F _in )

Among them, Pooling _x and Pooling _y represent the global average pooling along the x-axis direction and the y-axis direction, respectively;

Next, the point vectors on the compressed features F _x and F _y are used as graph nodes, and the adjacency matrixes _{A x} and A _y between the graph nodes on F x and F _y are respectively established:

和

分别表示F_x和F_y上的图节点空间；Among them, γ=100 is the scaling factor set by experience, dis( ) represents the Euclidean distance between two graph nodes, v _x,i ,v _x,j represent any two graph nodes obtained by F _x respectively, v _{y, i} , v _{y, j} respectively represent any two graph nodes obtained by F _y ,

and

denote the graph node spaces on F _x and F _y , respectively;

和F_y上图节点的邻接关系矩阵A_y的(i，j)位置像素点

求出相应的度矩阵

和

Then, according to the (i, j) position pixel point of the adjacency matrix A _x of the nodes in the above graph F _x and F _y

and the (i, j) position pixel point of the adjacency matrix A _y of the nodes in the graph above F _y

Find the corresponding degree matrix

and

和

分别对邻接关系矩阵A_x和A_y进行拉普拉斯归一化，得到归一化后的邻接关系矩阵L_x和L_y：Then, using the degree matrix

and

Perform Laplace normalization on the adjacency matrix A _x and A _y respectively to obtain the normalized adjacency matrix L _x and L _y :

Wherein, A' _x =A _x +I, A' _y =A _y +I, I is an identity matrix;

和

Finally, use the two-layer graph convolution to propagate the information between graph nodes to get the output of GCNB

and

Among them, σ( ) is the activation function, W _x and W′ _x are the learnable weight parameters in the x-axis direction, and W _y and W′ _y are the learnable weight parameters in the y-axis direction.

2.1.3) Establish fusion sub-module:

Referring to Figure 4, the fusion sub-module includes two activation layers and a convolution layer. After the two activation layers are connected in parallel, they are connected in series with the convolution layer. The working process is as follows:

Two activation layers connected in parallel are used to activate the two features output by GCNB to obtain two attention weights W _x and W _y :

得到初步融合特征，将初步融合特征经过一个卷积核大小为3×3的卷积层，得到融合特征

Multiply these two attention weights to the left and right of the output of CNNB respectively

Get the preliminary fusion features, pass the preliminary fusion features through a convolution layer with a convolution kernel size of 3×3, and obtain the fusion features

代表初步融合特征，⊙代表点乘操作；in,

Represents preliminary fusion features, ⊙ represents point multiplication operation;

2.1.4) The CNNB sub-module and the GCNB sub-module are connected in parallel, and then connected in series with the fusion sub-module to form a hybrid convolution unit, which is then connected to the other two pooling layers, that is, the first hybrid convolution unit , the first pooling layer, the second hybrid convolution unit, the second pooling layer, and the third hybrid convolution unit are sequentially cascaded to form a feature extraction module.

Through the feature extraction module, three features F ₁ , F ₂ , and F ₃ with different scales containing both local information and global information are obtained.

2.2) Establish a progressive fusion module:

The progressive fusion module fuses the three features F ₁ , F ₂ , and F ₃ obtained by the feature extraction module. It has three parallel branches, and each branch includes sequential sampling operations, splicing operations, and volume Product operation, on the three parallel branches, take F ₁ , F ₂ , and F ₃ as inputs respectively, and perform a feature fusion between F ₁ , F ₂ , and F ₃ . Taking the branch where the _F2 feature is located as an example, the process of its fusion once is described as follows:

2.2.1) Perform up-down sampling on features F ₁ and F ₃ of different sizes so that they are on the same size as F ₂ , and obtain corresponding up-down sampling features f ₁ and f ₃ :

f ₁ ＝Down(F ₁ )

f ₃ =Up(F ₃ )

Among them, Down( ) and Up( ) represent downsampling and upsampling operations, respectively;

2.2.2) Concatenate the three features f ₁ , F ₂ and f ₃ of the same size in the channel dimension to obtain the feature f _c after channel concatenation:

f _c =Concat[f ₁ , F ₂ , f ₃ ]

＝Concat[Down(F ₁ ),F ₂ ,Up(F ₃ )]

Among them, Concat[ ] represents the concatenation of channel dimensions;

2.2.3) Pass f _c sequentially through convolutional layers with convolution kernel sizes of 1×1, 3×3, and 1×1 to obtain the primary fusion feature F′ ₂ of the branch where the feature of F ₂ is located:

F′ ₂ ＝Conv _{1×1, 3×3, 1×1} (f _c )

Among them, Conv _{1×1, 3×3, 1×1} means three-layer convolution with convolution kernel size 1×1, 3×3, 1×1;

According to the process from 2.2.1) to 2.2.3), the primary fusion features F′ ₁ and F′ ₃ of the other two branches are obtained:

F′ ₁ ＝Conv _{1×1,3×3,1×1} (Concat[Up(F ₃ ),Up(F ₂ ),F ₁ ])

F′ ₃ ＝Conv _{1×1,3×3,1×1} (Concat[F ₃ ,Down(F ₂ ),Down(F ₁ )]);

2.2.4) Taking three primary fusion features F′ ₁ , F′ ₂ and F′ ₃ as input, according to the process from 2.2.1) to 2.2.3), respectively obtain the secondary fusion features F″ of the three branches ₁ , F″ ₂ and F″”:

F″ ₁ ＝Conv _{1×1,3×3,1×1} (Concat[Up(F′ ₃ ),Up(F′ ₂ ),F′ ₁ ])

F″ ₂ ＝Conv _{1×1,3×3,1×1} (Concat[Up(F′ ₃ ),F′ ₂ ,Down(F′ ₁ )])

F″ ₃ ＝Conv _{1×1,3×3,1×1} (Concat[F′ ₃ ,Down(F′ ₂ ),Down(F′ ₁ )]);

2.2.5) Take three secondary fusion features F″ ₁ , F″ ₂ and F″ ₃ as input, and follow the process from 2.2.1) to 2.2.3) to obtain the final fusion feature F:

F=Conv _{1×1, 3×3, 1×1} (Concat[Up(F″ ₃ ), Up(F″ ₂ ), F″ ₁ ]).

2.3) Create a label update module:

Referring to Figure 5, the construction of the tag update module is as follows:

2.3.1) Get the predicted label:

For the output F of the progressive fusion module, two convolutions with a kernel size of 1×1 are used to obtain a two-channel feature map, and the softmax activation function is used to obtain the predicted score of the feature map. After that, the predicted score is used in the channel dimension. max function to get the predicted label;

2.3.2) Get prototype tags:

For the predicted score in 2.3.1), based on the category of the predicted label, select the highest scoring k=1000 feature points in each category, and average the feature values of the k feature points of each category on the two-channel feature map to obtain Two category centers, and finally calculate the Euclidean distance from the feature value of each feature point on the two-channel feature map to the class center, assign the category label of the category center closest to each feature point to the feature point, and obtain the feature point Prototype label for location;

2.3.3) Update the test label and prototype label and the initial label obtained in step 1, and complete the creation of the label update module:

The initial label includes three categories of labels: unchanged category label, variable category label, and uncertain category label;

Both the predicted label and the prototype label only include the unchanged category and the changed category label;

The voting label is obtained from the prediction label and the prototype label according to the voting rules, that is, at the same coordinate position, if the prediction label is the same as the prototype label, the voting label is modified to be the same as the prediction label, and if the prediction label is different from the prototype label, then the The voting label is changed to an indeterminate category label;

Use the voting label to update the initial label according to the update rule, that is, at the same coordinate position, replace the uncertain category label in the initial label with the category label that changes and does not change in the voting label to obtain the final label.

Step 3: Use the three-channel input and label to train the GECN network.

The training of GECN is divided into two parts: pre-training and training.

The label includes an initial label used in the pre-training phase and an updated initial label used in the training phase;

3.1) Pre-train the GECN network:

3.1.1) Input the three-channel input constructed in 1.1) into GECN, and calculate the cross-entropy loss function L ₁ (θ) between the output of GECN and the initial label:

是GECN的输出，

h_θ(·)是具有参数θ的GECN，x是GECN的三通道输入；Among them, θ is the initial parameter of GECN obtained by random initialization, y ₁ is the initial label,

is the output of GECN,

h _θ ( ) is the GECN with parameter θ, x is the three-channel input of GECN;

3.1.2) Use the cross-entropy loss function for the output of GECN and the initial label, and use the gradient descent method to update the GECN parameter θ:

Set the initial pre-training batch m=0, and the maximum number of iterations in the pre-training stage T=20;

Calculate the GECN parameter θ _m+1 after the current pre-training update:

Among them, α is the learning rate in the pre-training stage, and θ _m is the network parameter before the current pre-training update;

3.1.3) Repeat 3.1.2) to make it reach the maximum number of iterations T of the pre-training stage, and complete the pre-training of GECN;

3.2) Retrain the pretrained GECN network:

3.2.1) Use the cross-entropy loss function for the output of the pre-trained GECN and the updated initial label, and calculate the cross-entropy loss function L ₂ of the output of the pre-trained GECN and the updated initial label:

L ₂ (θ')＝-[y ₂ logy'+(1-y ₂ )log(1-y')]

Among them, θ' is the network parameters after GECN pre-training, y ₂ is the updated initial label, y' is the GECN output after pre-training, y'=h _θ' (x'), h _θ' ( ) is GECN with parameters θ', x' is the three-channel input of GECN in the training phase;

3.2.2) Use the cross-entropy loss function for the output of the pre-trained GECN and the updated initial label, and use the gradient descent method to update the parameter θ' of the pre-trained GECN:

Set the initial training batch m'=0, and the maximum number of iterations T'=50 in the training phase;

Calculate the updated GECN parameters θ' _m+1 for the current training:

Among them, the learning rate of the training phase of α', θ' _m is the network parameter before the current training update;

3.2.3) Repeat 3.2.2) until the maximum number of iterations T' of the training phase is reached, and the trained GECN is obtained.

Step 4: Obtain the change result map.

4.1) Input the tested SAR image into the trained GECN to get the GECN output, use the max function on the channel dimension for the GECN output, and get the channel Channel where the maximum value is located:

Among them, x ₀ and x ₁ are the values on channel 0 and channel 1 of the GECN output feature, respectively;

4.2) According to the channel where the maximum value of each position on the GECN output feature corresponds to the attribute of the category of the pixel sample at the position, the category of each pixel sample can be obtained, that is, Channel=0 means that the pixel sample belongs to the unchanged category, Channel=1 means that the pixel point sample belongs to the change category, and the change detection of the SAR image is completed.

Effect of the present invention can be further illustrated by following simulation:

1. Simulation conditions

The simulation environment of the present invention selects the framework of python 3.8+pytorch 1.7, and completes it on the workstation of GeForce RTX2080Ti and 11G internal memory.

The four data sets used in the simulation are Bern data set, Ottawa data set, Yellow River I data set and Yellow River IV data set, among which,

The Bern dataset was captured by the European Remote Sensing 2 satellite sensor in April and May 1999, and the image size is 301×301;

The Ottawa dataset was captured by radar satellite sensors in May and August 1997, with an image size of 290×350;

The Yellow River I dataset and the Yellow River IV dataset are two classic areas of the Yellow River dataset, which were captured by the Radarsat-2 satellite sensor in June 2008 and June 2009, respectively, with image sizes of 289×257 and 306×291.

2. Simulation content

Under the above simulation conditions, use the present invention and the existing four methods DDNet, SAFNet, MsCapsNet, and MSDC to perform change detection simulation on four data sets, the visualization results are shown in Figure 6, and the numerical results are shown in Table 1 to Table 4 shows. in:

Figure 6(a) is the front-time image of the four datasets;

Figure 6(b) is the post-temporal image of the four datasets;

Figure 6(c) is the ground truth for the four datasets;

Figure 6(d) is the visualization result of the existing method DDNet on four datasets;

Figure 6(e) is the visualization result of the existing method SAFNet on four datasets;

Figure 6(f) is the visualization result of the existing method MsCapsNet on four datasets;

Figure 6(g) is the visualization result of the existing method MSDC on four datasets;

Fig. 6 (h) is the visualization result of the present invention on four data sets;

It can be seen from Fig. 6 that the present invention can effectively reduce the generation of noise points, and the boundary detection of changing objects is more accurate, which fully demonstrates the superiority of the method proposed by the present invention.

In order to quantitatively illustrate the performance of the proposed method, the numerical performance indicators commonly used in the change detection task are selected to measure the difference between the above existing methods and the present invention. The performance indicators include the overall error OE, the correct classification percentage PCC and the Kappa coefficient KC. The results are as follows table, where:

Table 1 is the numerical results of the present invention and four existing methods on the Bern dataset;

Table 2 is the numerical results of the present invention and four existing methods on the Ottawa dataset;

Table 3 is the numerical results of the present invention and four existing methods on the Yellow Rive I data set;

Table 4 is the numerical results of the present invention and four existing methods on the Yellow Rive IV data set;

Table 1 Bern dataset results

Table 2 Ottawa dataset results

Table 3 Yellow Rive I dataset results

Table 4 Yellow Rive IV dataset results

It can be seen from Table 1 to Table 4 that the overall error of the present invention is smaller and the change detection accuracy is higher, which further illustrates the superiority of the method proposed by the present invention.

The source of the four prior art:

DDNet is a method for SAR image change detection published by Qu Xiaofan et al. on IEEE, namely: X.Qu, F.Gao, J.Dong, Q.Du, and H.-C.Li, "Change detection in synthetic aperture radarimages using adult-domain network,” IEEE Geoscience and Remote Sensing Letters, vol.19, pp.1–5, 2021;

SAFNet is a method for SAR image change detection published by Gao Yunhao et al. in IEEE, namely: Y.Gao, F.Gao, J.Dong, Q.Du, and H.-C.Li, "Synthetic aperture radar image changedetection via siamese adaptive fusion network,"IEEE Journal of SelectedTopics in Applied Earth Observations and Remote Sensing,vol.14,pp.10 748–10760,2021;

MsCapsNet is a method for SAR image change detection published by Gao Yunhao et al. in IEEE, namely: Y.Gao, F.Gao, J.Dong, and H.-C.Li, "SAR image change detection based on multiscalecapsule network, "IEEE Geoscience and Remote Sensing Letters, vol.18, no.3, pp.484–488, 2020;

MSDC is a method for SAR image change detection published by Dong Huihui and others in IEEE, namely: Dong H, Ma W, Jiao L, et al. A multiscale self-attention deep clustering for change detection in SAR images[J].IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:1-16.

Claims (6)

1. a kind of SAR image change detection method based on hybrid convolution, is characterized in that, comprises the steps: (1) Construct three-channel input and initial label: 1a) Use the logarithmic ratio operator to obtain the difference map for the dual-time SAR image, and stitch the difference image and the dual-time image in the channel dimension to obtain a three-channel input; 1b) Clustering the difference map in 1a) using a fuzzy clustering algorithm to obtain an initial label; (2) Build a graph convolution enhanced convolutional network GECN: 2a) Establishing a feature extraction module comprising a convolutional neural network submodule CNNB, a graph convolutional neural network submodule GCNB and a fusion submodule, wherein CNNB and GCNB are connected in parallel and then connected in series with the fusion submodule; 2b) Establish a progressive fusion module that sequentially performs up-and-down sampling operations, splicing operations, and convolution operations; 2c) Establish a label update module that updates the training label by voting on the predicted label and the prototype label; 2d) The feature extraction module, progressive fusion module and label update module are connected in series to form a graph convolution enhanced convolutional network GECN; (3) Use three-channel input and label to train GECN: 3a) In the initial 20 training batches, the input of the three channels is sent to GECN, the output of GECN and the initial label are applied to the cross-entropy loss function, and the parameters of GECN are updated using the gradient descent method to complete the pre-training of GECN; 3b) In the next 50 training batches, the three-channel input is sent to the pre-trained GECN, the initial label is updated using the label update module, and cross-entropy is used for the output of the pre-trained GECN and the updated initial label Loss function, and use the gradient descent method to further update the pre-trained GECN parameters to complete the GECN training; (4) Input the tested SAR image into the trained GECN to get the GECN output, use the max function on the GECN output to get the category of each pixel sample on the SAR image, which is the change detection result of the final SAR image. 2. method according to claim 1, it is characterized in that, step 1b) uses fuzzy clustering algorithm to carry out clustering to difference map, realizes as follows: 1b1) Determine the number c of fuzzy clustering categories and the number of iterations k; 1b2) Initialize the membership matrix u with a random number between 0 and 1, so that the sum is 1, and the formula is described as:

Among them, u _ij is the membership degree of the j-th pixel sample belonging to the i-th category, and n is the number of pixel samples; 1b3) Calculate the cluster center according to the degree of membership:

Among them, c _i is the i-th cluster center, x _j is the j-th pixel sample; 1b4) Calculate the sum of squared errors J from the pixel point sample to various center points:

1b5) Find the extreme value of 1b4) formula, when the error sum of squares is minimum, get the updated membership degree matrix u _ij ; 1b6) Repeat 1b3) to 1b5) until the number of iterations k is reached to obtain the initial label. 3. The method according to claim 1, wherein the graph volume neural network submodule GCNB in step 2a) includes two parallel pooling layers and 4 graph convolution layers connected in sequence: The two parallel pooling layers are used to pool the input of GCNB along the x-axis direction and the y-axis direction respectively, so as to obtain graph nodes by compressing input features; The four graph convolution layers are connected in series and then connected in parallel, and are used to transfer the relationship between graph nodes to obtain two output features. 4. The method according to claim 1, characterized in that, the fusion submodule in step 2a) comprises two activation layers connected in parallel and a convolutional layer connected in turn: The two parallel-connected activation layers are used to first activate the two features output by GCNB to obtain attention weights, and then multiply the two attention weights to the left and right of the output of CNNB to obtain fusion features; The convolutional layer is used to further mine the information of the fusion feature to obtain the output features of the fusion sub-module. 5. The method according to claim 1, characterized in that, in step 3a), the output of GECN and the initial label adopt cross-entropy loss function, and use the gradient descent method to update the GECN parameters, as follows: 3a1) Calculate the cross-entropy loss function L ₁ of the output of GECN and the initial label:

where _y1 is the initial label,

is the output of GECN,

h _θ ( ) is the GECN with parameter θ, x is the three-channel input of GECN; 3a2) Update GECN parameters θ using gradient descent: Set the initial pre-training batch m=0, θ ₀ is the initial parameter obtained by random initialization, and the maximum number of iterations in the pre-training stage is T=20; Calculate the GECN parameter θ _m+1 after the current pre-training update:

Among them, α is the learning rate; 3a3) Repeat 3a2) to make it reach the maximum number of iterations T of the pre-training stage, and complete the pre-training of GECN. 6. The method according to claim 1, wherein, in step 3b), the output of the pre-trained GECN and the updated initial label adopt a cross-entropy loss function, and use the gradient descent method to update the pre-trained GECN parameters , implemented as follows: 3b1) Calculate the cross-entropy loss function L ₂ of the output of GECN after pre-training and the updated initial label: L ₂ (θ')＝-[y ₂ log y'+(1-y ₂ )log(1-y')] where, _y2 is the updated initial label, y' is the pre-trained GECN output, y'=h _θ' (x'), h _θ' ( ) is the GECN with parameters θ', x' is the trained Three-channel input to stage GECN; 3b2) Use the gradient descent method to update the parameters θ' of the GECN after pre-training: Set the initial training batch m'=0, _θ'0 is the parameter obtained after pre-training GECN, and the maximum number of iterations in the training phase is T'=50; Calculate the updated GECN parameters θ' _m+1 for the current training:

3b3) Repeat 3b2) until the maximum number of iterations T' of the training phase is reached, and a trained GECN is obtained.

Images