CN115761502A

CN115761502A - SAR image change detection method based on hybrid convolution

Info

Publication number: CN115761502A
Application number: CN202211472616.XA
Authority: CN
Inventors: 唐旭; 李德生; 马晶晶; 张向荣; 焦李成
Original assignee: Xidian University
Current assignee: Xidian University
Priority date: 2022-11-17
Filing date: 2022-11-17
Publication date: 2023-03-07

Abstract

The invention discloses a SAR image change detection method based on mixed convolution, which mainly solves the problems that the prior art is difficult to extract the global information of an image and lacks a reliable label training network under the condition of no label, and the scheme is as follows: constructing three-channel input and initial labels; building a graph convolution enhanced convolutional network (GECN) formed by cascading a feature extraction module, a progressive fusion module and a label updating module; training the GECN by using three-channel input and labels; and inputting the tested SAR image into the trained GECN to obtain a change detection result. According to the invention, the multi-scale features of the local and global information of the SAR image are obtained through the feature extraction module, so that the feature extraction capability of the GECN network is improved; more reliable labels are obtained through the label updating module to train the GECN, the generalization performance of the GECN is improved, and the method can be used for urban planning layout, natural disaster assessment and military dynamic reconnaissance.

Description

SAR image change detection method based on hybrid convolution

Technical Field

The invention belongs to the technical field of image processing, and relates to a method for detecting the change of a synthetic aperture radar SAR image, which can be used for urban planning layout, natural disaster assessment and military dynamic reconnaissance.

Background

The detection of changes in SAR images plays a crucial role in the field of remote sensing and is receiving increasing attention for its widespread use. The existing SAR image change detection technology is mainly divided into two types.

The first type is a change detection method based on a conventional algorithm, which detects a change by a threshold value with a characteristic difference and a ratio of a pixel pair or an object pair as an input in a region identification stage. The image processing unit can be further classified into an object-based method and a pixel-based method according to its size. However, in both the pixel-based and the object-based change detection methods, only manually extracted features, such as spectral features, texture features, and shape features, are used to extract change information, and these features cannot sufficiently represent key information of the SAR image, thereby greatly affecting the change detection result.

The second type is change detection based on a deep learning algorithm, and the method uses strong nonlinear fitting capability and a multi-level structure of a convolutional neural network, so that the learned characteristic features have high-level semantic information and rich spatial context information, and the change detection method is widely used at present. Although the change detection method based on deep learning has achieved great success in the change detection task of the SAR image, some problems still remain to be further solved. Firstly, the features of the SAR image are extracted only by using a convolutional neural network with a multilayer structure, theoretically, the deep features have global context information, but actually, the receptive field of the deep features is not global, and meanwhile, the global information cannot be effectively utilized on the shallow features, so that the features of the SAR image are not modeled by the global information. Secondly, generally, the fusion of the SAR multi-scale features is to directly add or splice channels after up-and-down sampling to the same size, however, in practice, the multi-scale features cannot be well fused through only one-time processing, so that the features are biased to a certain scale feature, and the multi-scale features are not fully utilized. Finally, most unsupervised change detection algorithms use clustering to obtain trained pseudo labels, so that a large number of wrong labels are mixed in the selected training labels, the reliability is low, and the network training cannot achieve the expected effect.

Sanyu is found in Scientific Citation Index (SCI) 2021,109, 107598, in the published heterogeneous remote sensing change detection paper based on non-local patch similarity, a graph (NPSG) based on non-local patch similarity is proposed to measure the structural consistency between heterogeneous images, thereby completing the change detection task. The concrete implementation is as follows: firstly, dividing an image into a series of patches, calculating k-nearest NPSG in an image before an event by utilizing similarity measurement based on statistics for each target patch in the image before the event in the forward detection, mapping the nearest NPSG with the image after the event, and comparing the nearest NPSG of the image after the event with the nearest NPSG of the image before the event by calculating a similarity difference; then, a threshold segmentation method is adopted for the comparison result to obtain a change detection result. The method adopts a k-nearest neighbor graph structure to model the global context of the target patch, and although better change detection precision can be obtained, a large amount of computing resources are consumed when the similarity between different patches is computed.

The Dong Hui et al, institute of Electrical and electronics Engineers IEEE,2021,60, have proposed a SAR image change detection method based on multi-scale self-attention depth clustering MSDC. Firstly, reducing speckle noise in an SAR image by using a side window filtering algorithm; then, taking the multi-scale adjacent area of the given central pixel as a basic difference analysis unit of the sample for the processed image; and then, fusing the multi-scale features to obtain the difference features of the double branches, and finally, optimizing the network by adopting the clustering-obtained label. In the process of multi-scale feature fusion, firstly, convolution nonlinear transformation is used for adjusting three scales of features to the same size, and then the three features with the same size are considered as a feature obtained by a self-attention mechanism through query q, a key value k and a numerical value v in a self-attention module; and finally, adding the features obtained by self attention and the features before self attention to complete the fusion of the multi-scale features. The ablation experiment of the board of coma proves the effectiveness of multi-scale attention, but the generation of the weight matrix in the self-attention mechanism needs a great amount of computing resources to support, so that the computing cost is too high and the application is inconvenient.

Dianthus superbus et al published a method for detecting changes of SAR images by using a double-domain network DDNet in IEEE,2021,19, which utilizes frequency domain and spatial domain feature representation to alleviate speckle noise, obtains tags by using hierarchical fuzzy clustering, and selects a 10% reliable tag training network according to membership degree. Although the method can effectively relieve the number of wrong labels, the number of labels participating in training is small, and further improvement of network performance is hindered.

Disclosure of Invention

The invention aims to provide a hybrid convolution-based SAR image change detection method aiming at the defects of the prior art, so as to extract multi-scale features with global information and local information, improve the detection precision and reduce the calculation resources.

In order to achieve the purpose, the technical scheme of the invention comprises the following steps:

(1) And (3) building a graph convolution reinforced convolution network GECN:

1a) Establishing a feature extraction module consisting of a convolutional neural network sub-module CNNB, a graph volume neural network sub-module GCNB and a fusion sub-module, wherein the CNNB is connected with the GCNB in parallel and then connected with the fusion sub-module in series;

1b) Establishing a progressive fusion module which sequentially performs up-down sampling operation, splicing operation and convolution operation;

1c) Establishing a label updating module for updating training labels by voting of network predicted labels and prototype labels;

1d) Connecting the feature extraction module, the progressive fusion module and the label updating module in series to form a graph convolution enhanced convolution network GECN;

(2) Get input and initial tag of GECN:

2a) Obtaining a difference image by adopting a logarithmic ratio operator for the double-time SAR image, and splicing the difference image and the double-time image in a channel dimension to obtain the input of a three-channel GECN;

2b) Clustering the difference graph in the step 2 a) by using a fuzzy clustering algorithm to obtain an initial label;

(3) Training the GECN with its inputs and labels:

3a) In the initial 20 training batches, three-channel input is sent into the GECN, cross entropy loss functions are adopted for the output of the GECN and the initial label, and a gradient descent method is used for updating GECN parameters to finish pre-training of the GECN;

3b) In the following 50 training batches, the three-channel input is sent to the pre-trained GECN, the initial label is updated by using a label updating module, the cross entropy loss function is adopted for the pre-trained GECN output and the updated initial label, and the pre-trained GECN parameter is further updated by using a gradient descent method, so that the training of the GECN is completed;

(4) And inputting the tested SAR image into the trained GECN to obtain GECN output, and obtaining the category of each pixel point sample on the SAR image by using a max function for the GECN output, namely the final SAR image change detection result.

Compared with the prior art, the invention has the following advantages:

1) According to the invention, the point vectors on the compression characteristics are taken as graph nodes by the GCNB, and graph convolution operation is respectively carried out along the x-axis direction and the y-axis direction, so that the global information of the image can be obtained under the condition of little information loss, the operation is simple, and the global information of the image can be fully mined.

2) The invention adopts a self-adaptive fusion mode to fuse the local and global information of the image, so that the information useful for the change detection task can be more fully explored.

3) Because the difference map is embedded into the input, the invention is equivalent to the prior information for GECN training, thereby ensuring that the GECN is converged more quickly and reducing the occupation of computing resources in the training process.

4) The invention adopts a mode of gradual multi-time fusion, fully explores multi-scale information of the image and avoids the problem that the characteristic is biased to a certain scale characteristic.

5) The invention uses the label updating module to update the initial label, obtains more reliable labels, and uses the labels to train the GECN, thereby increasing the generalization of the GECN and improving the change detection precision of the GECN.

Simulation experiments show that the invention can obtain excellent change detection performance.

Drawings

In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings used in the detailed description or the prior art description will be briefly described below.

FIG. 1 is a flow chart of an implementation of the present invention;

FIG. 2 is a GECN framework diagram of the graph convolution enhanced convolution network in the invention;

FIG. 3 is a schematic diagram of a GCNB structure of a graph convolution neural network sub-module of the GECN network in the invention;

FIG. 4 is a schematic diagram of the fusion submodule structure of the GECN network in the present invention;

FIG. 5 is a schematic diagram of a tag update module in the GECN network of the present invention;

fig. 6 is a diagram of simulation results of SAR image change detection using the present invention and the existing detection method, respectively.

Detailed Description

The examples and effects of the present invention will be described in further detail below with reference to the accompanying drawings.

Referring to fig. 1, the hybrid convolution-based SAR image change detection method in this embodiment includes four parts, namely, acquiring three-channel input and initial labels, building a graph convolution enhanced convolutional network GECN, training the GECN by using the three-channel input and labels, and acquiring a change result graph, and is specifically implemented as follows:

the method comprises the following steps: and acquiring three-channel input and an initial label.

1.1 Build three-channel input:

suppose two registered SAR images I acquired at the same place and different times ₁ And I ₂ To identify I ₁ And I ₂ The influence of speckle noise in the SAR image on the change detection performance is reduced, and the change detection performance is firstly influenced by the image I ₁ And I ₂ Smoothing speckle noise by using a logarithmic ratio operator to obtain a difference image D; to compensate for the loss of information in D due to the smoothing effect of the logarithmic ratio operator, I ₁ 、I ₂ And splicing the difference graph D in the channel dimension to obtain the input of three channels.

1.2 Obtain initial tag:

clustering the difference graph D in the step 1.1) by using a fuzzy clustering algorithm, wherein the working process is as follows:

1.2.1 Determining the number c of fuzzy clustering categories and the iteration number t;

1.2.2 ) initialize a membership matrix u with random numbers having values between 0 and 1, with the sum 1:

wherein u is _ij The membership degree of the jth pixel point sample belonging to the ith category, and n is the number of the pixel point samples;

1.2.3 Compute cluster centers based on membership:

wherein, c _i Is the ith cluster center, x _j Is the jth pixel sample;

1.2.4 Calculating the square sum J of errors from pixel point samples to various central points:

1.2.5 Solving an extreme value of the formula of 1.2.4), and obtaining an updated membership matrix u when the sum of squared errors is minimum _ij ；

1.2.6 Repeat 1.2.3) to 1.2.5) until the number of iterations t is reached, resulting in the initial label.

And step two, constructing a graph convolution enhanced convolution network GECN.

Referring to fig. 2, the gecn includes a feature extraction module, a progressive fusion module, and a tag update module connected in series, and the building process of each module is as follows:

2.1 Establishing a feature extraction module comprising a convolutional neural network sub-module CNNB, a graph convolutional neural network sub-module GCNB and a fusion sub-module:

2.1.1 Three convolution layers with convolution kernel sizes of 1 × 1, 3 × 3 and 1 × 1 are sequentially connected to form a CNNB sub-module for exploring local information of an image;

2.1.2 Create GCNB submodule:

referring to fig. 3, the GCNB sub-module includes two parallel pooling layers and four graph convolution layers, the four graph convolution layers are connected in series two by two and then connected with each pooling layer respectively, and the working process of the GCNB sub-module is as follows:

first, the input F to the GCNB is from two parallel pooling layers along the x-axis direction and the y-axis direction, respectively _in Global average pooling is carried out to obtain a compressed characteristic F _x And F _y ：

F _x ＝Pooling _x (F _in )

F _y ＝Pooling _y (F _in )

Wherein Pooling _x And Pooling _y Representing global planes in the x-axis direction and the y-axis direction, respectivelyHomogenizing and pooling;

then, the compressed feature F _x And F _y Taking the point vectors as graph nodes, and respectively establishing F _x And F _y Adjacency relation matrix A between nodes of upper graph _x And A _y ：

Where γ =100 is an empirically set scaling factor, dis (·) represents the euclidean distance between two graph nodes, v · _x,i ,v _x,j Each represents any two of F _x Resulting graph nodes, v _y,i ,v _y,j Each represents any two of F _y The nodes of the obtained graph are connected with each other,

and

respectively represent F _x And F _y An upper graph node space;

then, according to F _x And F _y Adjacency relation matrix A of nodes of upper graph _x (i, j) position pixel point

And F _y Adjacency relation matrix A of nodes of upper graph _y (i, j) position pixel point of (c)

Determining a corresponding degree matrix

And

then, the degree of use matrix

And

respectively to the adjacency relation matrix A _x And A _y Performing Laplace normalization to obtain a normalized adjacency relation matrix L _x And L _y ：

Wherein, A' _x ＝A _x +I，A' _y ＝A _y + I, I is the identity matrix;

finally, information between nodes of the propagation graph is convolved by using the two-layer graph to obtain the output of the GCNB

And

where σ (-) is the activation function, W _x And W' _x Is a learnable weight parameter in the x-axis direction, W _y And W' _y Is a learnable weight parameter in the y-axis direction.

2.1.3 Create a fusion submodule:

referring to fig. 4, the fusion submodule includes two active layers and a convolution layer, and after the two active layers are connected in parallel, the two active layers are connected in series with the convolution layer, and the working process is as follows:

two parallel connected activation layers for activating two features of the GCNB output, resulting in two attention weights W _x And W _y ：

Multiplying the two attention weights to the left and right of the output of the CNNB, respectively

Obtaining a preliminary fusion feature, and passing the preliminary fusion feature through a convolution layer with convolution kernel size of 3 multiplied by 3 to obtain the fusion feature

Wherein,

represents a preliminary fusion feature, <' > represents a point productOperating;

2.1.4 Respectively connecting the CNNB sub-module and the GCNB sub-module in parallel, then connecting the CNNB sub-module and the GCNB sub-module in series to form a mixed convolution unit, and then connecting the mixed convolution unit with other two pooling layers, namely, sequentially connecting the first mixed convolution unit, the first pooling layer, the second mixed convolution unit, the second pooling layer and the third mixed convolution unit in series to form a feature extraction module.

Through the feature extraction module, three features F which simultaneously contain local information and global information and have different scales are obtained ₁ 、F ₂ 、F ₃ 。

2.2 Build a progressive fusion module:

the progressive fusion module is used for extracting three characteristics F obtained by the characteristic extraction module ₁ 、F ₂ 、F ₃ Merging, namely, merging three branches connected in parallel, wherein each branch comprises sampling operation, splicing operation and convolution operation which are sequentially performed, and F is respectively used for the three branches connected in parallel ₁ 、F ₂ 、F ₃ For input, perform F ₁ 、F ₂ 、F ₃ One feature fusion in between. Following with F ₂ The process of fusing the branch with the characteristic as an example is described as follows:

2.2.1 For different size features F) ₁ 、F ₃ Sampling up and down to make it same as F ₂ On the same size, obtaining corresponding up-down sampled characteristic f ₁ And f ₃ ：

f ₁ ＝Down(F ₁ )

f ₃ ＝Up(F ₃ )

Wherein, down (-) and Up (-) represent Down sampling and Up sampling operations, respectively;

2.2.2 Three same-size features f) ₁ 、F ₂ And f ₃ Performing channel dimension splicing to obtain the characteristic f after channel splicing _c ：

f _c ＝Concat[f ₁ ，F ₂ ，f ₃ ]

＝Concat[Down(F ₁ ),F ₂ ,Up(F ₃ )]

Wherein Concat [. Cndot. ] represents the concatenation of channel dimensions;

2.2.3 Is f) mixing _c Sequentially passing through convolution layers with convolution kernel sizes of 1 × 1, 3 × 3 and 1 × 1 to obtain F ₂ Primary fusion feature F 'of branch of feature' ₂ ：

F′ ₂ ＝Conv _{1×1，3×3，1×1} (f _c )

Wherein, conv _{1×1,3×3,1×1} Three-layer convolution representing convolution kernel sizes of 1 × 1, 3 × 3 and 1 × 1;

according to the processes of 2.2.1) to 2.2.3), obtaining a primary fused feature F 'of the other two branches' ₁ And F' ₃ ：

F′ ₁ ＝Conv _{1×1,3×3,1×1} (Concat[Up(F ₃ ),Up(F ₂ ),F ₁ ])

F′ ₃ ＝Conv _{1×1,3×3,1×1} (Concat[F ₃ ,Down(F ₂ ),Down(F ₁ )])；

2.2.4 Three primary fused feature F' ₁ 、F′ ₂ And F' ₃ For inputting, according to the processes from 2.2.1) to 2.2.3), respectively obtaining the secondary fusion characteristics F ″' of the three branches ₁ 、F″ ₂ And F':

F″ ₁ ＝Conv _{1×1,3×3,1×1} (Concat[Up(F′ ₃ ),Up(F′ ₂ ),F′ ₁ ])

F″ ₂ ＝Conv _{1×1,3×3,1×1} (Concat[Up(F′ ₃ ),F′ ₂ ,Down(F′ ₁ )])

F″ ₃ ＝Conv _{1×1,3×3,1×1} (Concat[F′ ₃ ,Down(F′ ₂ ),Down(F′ ₁ )])；

2.2.5 With three secondary fusion features F ″) ₁ 、F″ ₂ And F ″) ₃ For input, according to the process from 2.2.1) to 2.2.3), the final fusion feature F is obtained:

F＝Conv _{1×1,3×3,1×1} (Concat[Up(F″ ₃ ),Up(F″ ₂ ),F″ ₁ ])。

2.3 Build tag update module:

referring to fig. 5, the tag update module is constructed as follows:

2.3.1 Get a predictive tag:

performing convolution with the kernel size of 1 multiplied by 1 on the output F of the progressive fusion module to obtain a feature map of two channels, using a softmax activation function to obtain a prediction score of the feature map, and then using a max function on the prediction score in a channel dimension to obtain a prediction label;

2.3.2 Get prototype tag:

for the predicted score in 2.3.1), based on the category of the predicted label, selecting k =1000 feature points with the highest score in each category, averaging the feature values of k feature points in each category on two channel feature maps to obtain two category centers, finally calculating the Euclidean distance from the feature value of each feature point on the two channel feature maps to the category center, assigning the category label of the category center with the nearest feature point distance to the feature point, and obtaining a prototype label of the feature point position;

2.3.3 Updating the test label, the prototype label and the initial label obtained in the step one to complete the creation of a label updating module:

the initial labels comprise three types of labels including an invariant type label, a variant type label and an uncertain type label;

the prediction label and the prototype label only comprise an unchanged category label and a changed category label;

obtaining a voting label by the prediction label and the prototype label according to a voting rule, namely in the same coordinate position, modifying the voting label to be the same as the prediction label if the prediction label is the same as the prototype label, and modifying the voting label to be an uncertain category label if the prediction label is not the same as the prototype label;

and updating the initial label by using the voting label according to an updating rule, namely replacing the uncertain category label in the initial label by using the category label which is changed or not changed in the voting label at the same coordinate position to obtain a final label.

Step three: the GECN network is trained using three-channel inputs and labels.

The training of the GECN is divided into two parts of pre-training and training,

the labels comprise an initial label used in a pre-training stage and an updated initial label used in a training stage;

3.1 Pre-training the GECN network:

3.1.1 Inputting the three-channel input constructed in 1.1) into the GECN, and calculating a cross entropy loss function L of the output of the GECN and the initial label ₁ (θ)：

Where θ is the GECN initial parameter obtained by random initialization, y ₁ Is the initial label of the tag that is to be used,

is the output of the GECN and is,

h _θ (. Cnd) is the GECN with parameter θ, x is the three-channel input of the GECN;

3.1.2 Apply a cross entropy loss function to the GECN output and initial label, update the GECN parameter θ using a gradient descent method:

setting an initial pre-training batch m =0, and setting the maximum iteration number T =20 in a pre-training stage;

calculating the updated GECN parameter theta of the current pre-training _m+1 ：

Where α is the learning rate of the pre-training phase and θ _m Is the network parameter before the current pre-training update;

3.1.3 3.1.2) is repeated to reach the maximum iteration number T of the pre-training stage, and the pre-training of the GECN is completed;

3.2 Retraining the pre-trained GECN network:

3.2.1 Adopting cross entropy loss function to output of the pre-trained GECN and the updated initial label, and calculating cross entropy loss function L of the output of the pre-trained GECN and the updated initial label ₂ ：

L ₂ (θ')＝-[y ₂ logy'+(1-y ₂ )log(1-y')]

Where θ' is the network parameter after GECN pre-training, y ₂ Is the updated initial label, y 'is the pre-trained GECN output, y' = h _θ' (x')，h _θ' (. H) is the GECN with parameter θ ', x' is the three-channel input to the training phase GECN;

3.2.2 ) a cross entropy loss function is adopted for the output of the pre-trained GECN and the updated initial label, and a gradient descent method is used for updating a parameter theta' of the pre-trained GECN:

setting an initial training batch m '=0 and the maximum iteration number T' =50 in a training stage;

calculating the updated GECN parameter theta 'of the current training' _m+1 ：

Wherein, the learning rate of the training phase of alpha ', theta' _m Is the network parameter before the current training update;

3.2.3 3.2.2) is repeated until the maximum number of iterations T' of the training phase is reached, resulting in a trained GECN.

Step four: and obtaining a change result graph.

4.1 Input the tested SAR image into the trained GECN to obtain GECN output, and obtain the Channel where the maximum value is located by using max function for the output of the GECN in the Channel dimension:

wherein x is ₀ And x ₁ The values on channel 0 and channel 1 of the GECN output characteristic, respectively;

4.2 According to the attribute that the Channel where the maximum value of each position on the GECN output characteristic is located corresponds to the category of the pixel point sample at the position, the category of each pixel point sample can be obtained, that is, channel =0 represents that the pixel point sample belongs to the unchanged category, and Channel =1 represents that the pixel point sample belongs to the changed category, so that the change detection of the SAR image is completed.

The effects of the present invention can be further illustrated by the following simulations:

simulation conditions

The simulation environment of the invention selects a frame of python 3.8+ pytorch 1.7 and completes the simulation on the workstations of GeForce RTX 2080Ti and 11G memories.

The four datasets used in the simulation were the Bern dataset, ottawa dataset, yellow River I dataset, and Yellow River IV dataset, respectively, wherein,

the Bern data set is captured by a European remote sensing No. 2 satellite sensor in 4 months and 5 months in 1999 respectively, and the image size is 301 multiplied by 301;

ottawa data sets were captured by radar satellite sensors in months 5 and 8, 1997, respectively, with image sizes of 290 × 350;

the Yellow River I and IV datasets are two classic regions of the Yellow River dataset, captured by the Radarsat-2 satellite sensor at 6 months 2008 and 6 months 2009, respectively, with image sizes of 289 × 257 and 306 × 291, respectively.

Second, simulation content

Under the above simulation conditions, change detection simulation was performed on four data sets by using DDNet, SAFNet, mspapsn, and MSDC according to the present invention and the four existing methods, respectively, and the visualization results are shown in fig. 6, and the numerical results are shown in tables 1 to 4. Wherein:

FIG. 6 (a) is a prior time image of four data sets;

FIG. 6 (b) is a time-after image of four data sets;

FIG. 6 (c) is a ground truth for four data sets;

FIG. 6 (d) is a visualization of prior art method DDNet on four datasets;

FIG. 6 (e) is a visualization of a prior art method SAFNet on four data sets;

fig. 6 (f) is a visualization of the prior art method mspapset on four datasets;

fig. 6 (g) is a visualization of the existing method MSDC on four data sets;

FIG. 6 (h) is a visualization of the present invention on four datasets;

as can be seen from fig. 6, the present invention can effectively reduce the generation of noise, and the boundary detection of the changed ground object is more accurate, which fully illustrates the superiority of the method provided by the present invention.

In order to quantitatively illustrate the performance of the proposed method, the commonly used numerical performance indexes of the change detection task are selected to measure the difference between the existing method and the present invention, and the performance indexes comprise the total error OE, the percentage PCC of correct classification and the Kappa coefficient KC, and the results are as follows, wherein:

table 1 is a numerical result on a Bern dataset for the present invention and four prior methods;

table 2 is the numeric results on Ottawa dataset for the present invention and four prior methods;

table 3 is a numerical result on the Yellow Rive I dataset for the present invention and four prior methods;

table 4 is a numerical result on the Yellow Rive IV dataset for the present invention and four prior methods;

TABLE 1 Bern data fruit set

TABLE 2 Ottawa data fruit staging

TABLE 3 Yellow Rive I data set fruit

TABLE 4 Yellow Rive IV data set fruit

As can be seen from tables 1 to 4, the overall error of the method is smaller, and the change detection precision is higher, further illustrating the superiority of the method provided by the invention.

The four prior art sources:

DDNet is a method published by fringed et al on IEEE for SAR image change detection, namely: X.Qu, F.Gao, J.Dong, Q.Du, and H.C.Li, "Change detection in synthetic approach using an addition-domain network," IEEE Geoscience and Remote Sensing Letters, vol.19, pp.1-5,2021;

SAFNet is a method published by high yunhao et al in IEEE for SAR image change detection, namely: Y.Gao, F.Gao, J.Dong, Q.Du, and H.C.Li, "Synthetic aperture radio image change detection motion adaptive fusion network," IEEE Journal of Selected Topics in Applied Earth emissions and removal Sensing, vol.14, pp.10-10 760,2021;

MsCapsNet is a method published by IEEE by high yunnan et al for SAR image change detection, namely: y.gao, f.gao, j.dong, and h.c.li, "SAR image change detection based on multiscale capsule network," IEEE Geoscience and Remote Sensing Letters, vol.18, no.3, pp.484-488,2020;

MSDC is a method for SAR image change detection published in IEEE by tuo hui et al, namely: dong H, ma W, jiano L, et al. A multiscale self-alignment for change detection in SAR images [ J ]. IEEE Transactions on geoccience and removal Sensing,2021, 60.

Claims

1. A SAR image change detection method based on mixed convolution is characterized by comprising the following steps:

(1) Constructing a three-channel input and initial label:

1a) Obtaining a difference image by adopting a logarithmic ratio operator for the double-time SAR image, and splicing the difference image and the double-time image in a channel dimension to obtain three-channel input;

1b) Clustering the difference graph in the step 1 a) by using a fuzzy clustering algorithm to obtain an initial label;

(2) And (3) building a graph convolution enhanced convolution network GECN:

2a) Establishing a feature extraction module consisting of a convolutional neural network sub-module CNNB, a graph convolutional neural network sub-module GCNB and a fusion sub-module, wherein the CNNB is connected with the GCNB in parallel and then connected with the fusion sub-module in series;

2b) Establishing a progressive fusion module for sequentially performing up-down sampling operation, splicing operation and convolution operation;

2c) Establishing a label updating module for updating training labels by voting of the predicted labels and the prototype labels;

2d) Connecting the feature extraction module, the progressive fusion module and the label updating module in series to form a graph convolution enhanced convolution network GECN;

(3) Training the GECN using three-channel inputs and labels:

2. The method according to claim 1, characterized in that the disparity map in step 1 b) is clustered using a fuzzy clustering algorithm, which is implemented as follows:

1b1) Determining the number c and the iteration times k of fuzzy clustering categories;

1b2) Initializing the membership matrix u with a random number having a value between 0 and 1 such that the sum is 1, the formula is described as:

1b3) Calculating a clustering center according to the membership:

wherein, c _i Is the ith cluster center, x _j Is the jth pixel sample;

1b4) Calculating the sum of the squares of the errors from the pixel point samples to various central points J:

1b5) Solving the extreme value of the formula 1b 4), and obtaining the updated membership matrix u when the sum of the squared errors is minimum _ij ；

1b6) Repeating 1b 3) to 1b 5) until the iteration number k is reached, and obtaining an initial label.

3. The method according to claim 1, wherein the graph volume neural network sub-module GCNB in step 2 a) comprises two parallel pooling layers and 4 graph volume layers connected in sequence:

the two parallel pooling layers are used for pooling the input of the GCNB along the x-axis direction and the y-axis direction respectively so as to obtain a graph node by compressing the input characteristics;

the 4 graph convolution layers are connected in parallel after being connected in series two by two and are used for transmitting the relationship between graph nodes to obtain two output characteristics.

4. The method of claim 1, wherein the fusion submodule in step 2 a) comprises two parallel-connected active layers and a convolutional layer connected in sequence:

the two activation layers connected in parallel are used for activating two features output by the GCNB to obtain attention weights, and then multiplying the two attention weights by the output of the CNNB to obtain fusion features;

the convolution layer is used for further mining the information of the fusion characteristic to obtain the output characteristic of the fusion sub-module.

5. The method according to claim 1, characterized in that in step 3 a) a cross entropy loss function is applied to the GECN output and the initial label, and the GECN parameters are updated using a gradient descent method, which is implemented as follows:

3a1) Calculating a cross entropy loss function L of the output of the GECN and the initial label ₁ ：

Wherein, y ₁ Is the initial label of the tag that is to be used,

is the output of the GECN and is,

h _θ (. O) is the GECN with parameter θ, x is the three-channel input of the GECN;

3a2) Updating the GECN parameter θ using a gradient descent method:

let initial pre-training batch m =0, θ ₀ For the initial parameters obtained by random initialization, the maximum iteration number T =20 in the pre-training stage;

Wherein α is the learning rate;

3a3) And repeating the step 3a 2) to enable the maximum iteration number T of the pre-training stage to be reached, and completing the pre-training of the GECN.

6. The method according to claim 1, wherein the cross entropy loss function is applied to the output of the pre-trained GECN and the updated initial label in step 3 b), and the gradient descent method is used to update the pre-trained GECN parameters, which is implemented as follows:

3b1) Calculating a cross entropy loss function L of the output of the pre-trained GECN and the updated initial label ₂ ：

L ₂ (θ')＝-[y ₂ log y'+(1-y ₂ )log(1-y')]

Wherein, y ₂ Is the updated initial label, y 'is the pre-trained GECN output, y' = h _θ' (x')，h _θ' (. H) is the GECN with parameter θ ', x' is the three-channel input to the training phase GECN;

3b2) Updating the parameter theta' of the pre-trained GECN by using a gradient descent method:

let initial training batch m '=0, θ' ₀ The maximum iteration times T' =50 in the training stage for the parameters obtained after the GECN is pre-trained;

calculating the updated GECN parameter theta 'of the current training' _m+1 ：

3b3) Repeating 3b 2) until the maximum iteration number T' of the training phase is reached, and obtaining the trained GECN.