CN117788379A - Domain transformation-based end-to-end heterogeneous remote sensing image change detection method - Google Patents
Domain transformation-based end-to-end heterogeneous remote sensing image change detection method Download PDFInfo
- Publication number
- CN117788379A CN117788379A CN202311587728.4A CN202311587728A CN117788379A CN 117788379 A CN117788379 A CN 117788379A CN 202311587728 A CN202311587728 A CN 202311587728A CN 117788379 A CN117788379 A CN 117788379A
- Authority
- CN
- China
- Prior art keywords
- network
- domain
- remote sensing
- image
- change detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000008859 change Effects 0.000 title claims abstract description 63
- 238000001514 detection method Methods 0.000 title claims abstract description 49
- 230000009466 transformation Effects 0.000 title claims abstract description 31
- 238000000034 method Methods 0.000 claims abstract description 25
- 238000013519 translation Methods 0.000 claims abstract description 5
- 230000004927 fusion Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims 1
- 230000001131 transforming effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 3
- 238000013508 migration Methods 0.000 description 3
- 230000005012 migration Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses an end-to-end heterogeneous remote sensing image change detection method based on domain transformation, which comprises the following steps: acquiring a heterogeneous remote sensing image pair, and dividing two images in the image pair into a plurality of paired subgraphs with the same size; constructing a domain transformation network by adjusting a loop generation countermeasure network that has been proposed for application to image translation; constructing a change detection network; inputting the segmented paired subgraphs into a domain transformation network so that the two images are in the same image domain; and inputting the image pairs in the same image domain into a change detection network to obtain a binary change prediction graph. The method is different from the traditional heterogeneous remote sensing change detection method, firstly, a domain transformation network is used for transforming remote sensing images in different image domains into the same domain, and then, the end-to-end detection is realized through a change detection network, so that the accuracy of the change detection is improved.
Description
Technical Field
The invention relates to the technical field of image processing and heterogeneous remote sensing image change detection, in particular to an end-to-end heterogeneous remote sensing image change detection method based on domain transformation.
Background
The remote sensing image change detection refers to a process of detecting a change region by analyzing remote sensing images from different points in time at the same place. The change detection is an important research direction in the remote sensing field, and plays a key role in various fields such as urban research, agricultural investigation, environmental condition monitoring, disaster loss evaluation and the like. The remote sensing image can now be acquired by various remote sensing sensors, such as: optical sensors, multispectral sensors, synthetic Aperture Radar (SAR), etc., each type of remote sensing image has its own advantages and disadvantages. In some complex scenarios it is necessary to process heterologous remote sensing images from different sensor acquisitions for analysis and evaluation. However, the properties of different types of remote sensing images are very different, and cannot be directly compared to detect changes. Therefore, the heterogeneous remote sensing image change detection is a hot spot for research in the remote sensing field.
Existing heterogeneous remote sensing change detection methods are few and often rely on manually extracting features of images. However, in practical application, a large amount of manual interference is needed, the efficiency is low, the characterization effect is not ideal, the quality of the generated change map is poor, and the accuracy of detecting the change of the heterogeneous remote sensing image is low. There are also methods for detecting heterogeneous remote sensing changes using deep neural networks, which use deep neural networks to extract nonlinear features of high dimensions of images. However, the two heterogeneous remote sensing images are directly processed by the method, different distribution characteristics among different types of remote sensing images are not considered, and the proposed network learning capability is poor.
Disclosure of Invention
The invention aims to provide an end-to-end heterogeneous remote sensing image change detection method based on domain transformation, which realizes heterogeneous remote sensing image change detection and improves the accuracy of change detection.
The technical scheme for realizing the purpose of the invention is as follows: an end-to-end heterogeneous remote sensing image change detection method based on domain transformation comprises the following steps:
step 1: acquiring a heterogeneous remote sensing image pair, and dividing two images in the image pair into a plurality of paired subgraphs with the same size;
step 2: constructing a domain transformation network by adjusting a loop generation countermeasure network that has been proposed for application to image translation;
step 3: constructing a change detection network;
step 4: inputting the segmented paired subgraphs into a domain transformation network so that the two images are in the same image domain;
step 5: and inputting the image pairs in the same image domain into a change detection network to obtain a binary change prediction graph.
Further, the sub-picture size in step 1 is 256×256.
Further, the domain transformation network constructed in the step 2 is specifically as follows:
the domain transformation network is a conditional generation countermeasure network, and has two generators G with the same structure 1 And G 2 Two identical-structure discriminators D 1 And D 2 . The generator network consists of a U-Net structure, a space transducer module and a time transducer module, and the discriminator network consists of a multi-scale discriminator.
Further, the constructed change detection network in step 3 is specifically as follows:
the change detection network has a dual-branch twin structure sharing weights, each branch consisting of one convolution block and three residual convolution blocks. The network also has three transformers modules for capturing long-distance dependencies and global dependencies and one feature fusion module.
Further, the process of step 4 is specifically as follows:
randomly initializing network parameters of two generators and two multi-scale discriminators; inputting two paired heterologous remote sensing images X and Y into a domain transformation network; the two transformed images X are obtained through the repeated countermeasure iteration training of the network 1 And Y 1 Wherein X and X 1 Belonging to the same image domain, Y and Y 1 Belonging to another identical image domain.
Further, the process of step 5 is specifically as follows:
randomly initializing network parameters of a change detection network; selecting a group of image input change detection networks which are easy to process according to the two groups of images in the same domain obtained in the step 4; through twin generationThree groups of multi-scale features F are obtained by structuring two images 1 ，F 2 ，F 3 Obtaining three multi-scale difference features D through element level difference 1 ，D 2 ，D 3 The method comprises the steps of carrying out a first treatment on the surface of the Three multi-scale difference features D 1 ，D 2 ，D 3 Respectively entering three transducer modules; output of three transducer modulesEntering a feature fusion module for fusion; and classifying the fused features to obtain a final binary change prediction graph.
Compared with the prior art, the method provides an end-to-end heterogeneous remote sensing image change detection method, and is characterized in that a domain transformation network is firstly constructed to transform different image domains into the same image domain in the face of a pair of heterogeneous remote sensing images which are difficult to process, a change detection network is further constructed, multi-scale features are extracted from two images in double time phases through a twin structure sharing weight, difference features are extracted through a transducer module, a binary change prediction graph is output after feature fusion, and the accuracy of heterogeneous remote sensing image change detection is improved.
Drawings
FIG. 1 is a flow chart of a method for detecting the change of a heterogeneous remote sensing image end to end based on domain transformation;
FIG. 2 is a schematic diagram of a generator in a domain migration network according to the present invention;
FIG. 3 is a schematic diagram of a discriminator in a domain transfer network according to the invention;
FIG. 4 is a schematic diagram of a change detection network according to the present invention;
fig. 5 is a diagram of a detection result according to an embodiment of the present invention.
Detailed Description
As shown in fig. 1, the specific steps of the implementation of the present invention are as follows:
step 1: and acquiring a heterogeneous remote sensing image pair, and dividing two images in the image pair into subgraphs with the same size. In one embodiment, the sub-graph size in step 1 is 256×256.
Step 2: a domain transformation network as shown in fig. 2 and 3 is constructed. The domain transformation network is a generating countermeasure network and consists of two generator networks and a discriminator network which are identical in structure. The method comprises the following steps:
step 2-1: the main structure of the generator network is U-Net, 4 stages are respectively arranged in the encoding stage and the decoding stage of the U-Net, the resolution of the feature map in each stage is 256, 128, 64 and 32, and the number of the feature map units is 48, 96, 192 and 384;
step 2-2: the feature map of the bottom layer is further input into a space transformer module of the bottom layer;
step 2-3: the multi-scale characteristics of 4 stages enter a multi-scale channel transducer module;
step 2-4: the space transformer module and the multi-scale channel transformer module output characteristic diagrams subjected to self-attention calculation;
step 2-5: and finally outputting the transformed image in the decoding stage of the U-Net.
Step 3: a change detection network as shown in fig. 4 is constructed. The method comprises the following steps:
step 3-1: the change detection network is provided with a double-branch twin structure sharing weight and is used for extracting multi-scale characteristics from two input images respectively. Each branch of the twinning network consists of one convolution block and three residual convolution blocks. The convolution block is set as a layer of convolution layer with the convolution kernel of 3×3, and the number of the feature map units is 24. The residual convolution block is set as a two-layer convolution layer with a convolution kernel of 3 multiplied by 3, and the number of the characteristic diagram units is 48, 96 and 192;
step 3-2: three residual convolution blocks of each branch of the twin structure can obtain three multi-scale characteristics F 1 ，F 2 ，F 3 Two groups of multi-scale features are subjected to element level difference to obtain difference features D 1 ，D 2 ，D 3 Is input into three transducer modules;
step 3-3: three differential features through a transducer moduleAnd obtaining a binary change prediction graph through feature fusion.
Examples
The experimental effect of the end-to-end heterogeneous remote sensing image change detection method based on domain transformation is further described by taking a data set of the Grosvenor region in British as a data set, and the method comprises the following specific steps:
step 1: the optical image acquired in 2006 and the SAR image acquired in 2007 in the Grosvenor region of the United kingdom are loaded, the two images and the corresponding label image are segmented, the resolution of the sub-images after segmentation is 256 multiplied by 256, and the data are divided into a training set and a testing set. As shown in fig. 5, 4 sets of subgraphs and detection results.
Step 2: constructing a domain migration network and initializing;
step 3: constructing a change detection network and initializing;
step 4: inputting the training set data in the step 1 into a domain migration network to obtain two groups of images in the same image domain
Step 5: and (3) inputting the image obtained in the step (4) into a change detection network to obtain a final binary change prediction graph, and setting a loss function such as cross entropy to minimize the loss between the binary change prediction graph and the label graph, thereby completing model training.
Step 6: and importing the parameters trained by the model, and outputting a comparison result through the test set. As shown in fig. 5, (a) is a SAR image, (b) is an optical image, (c) is a label image, (d) is a full convolution twin network detection result, (e) is a depth translation change detection network detection result, and (f) is a proposed method detection result. And calculating the accuracy, recall, F1 score and cross ratio to evaluate the detection result. The evaluation results are shown in table 1.
TABLE 1
| Accuracy (%) | Recall (%) | F1 fraction (%) | Cross ratio (%) | |
| Full convolution twin network | 89.08 | 91.01 | 89.52 | 82.35 |
| Deep translation change detection network | 88.83 | 89.17 | 88.99 | 80.32 |
| The method | 92.44 | 90.31 | 91.35 | 84.08 |
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (6)
1. The method for detecting the change of the end-to-end heterogeneous remote sensing image based on domain transformation is characterized by comprising the following steps of:
step 1: acquiring a heterogeneous remote sensing image pair, and dividing two images in the image pair into a plurality of paired subgraphs with the same size;
step 2: constructing a domain transformation network by adjusting a loop generation countermeasure network that has been proposed for application to image translation;
step 3: constructing a change detection network;
step 4: inputting the segmented paired subgraphs into a domain transformation network so that the two images are in the same image domain;
step 5: and inputting the image pairs in the same image domain into a change detection network to obtain a binary change prediction graph.
2. The method for detecting the change of the end-to-end heterogeneous remote sensing image based on the domain transformation according to claim 1, wherein the pair sub-graph size in the step 1 is 256×256.
3. The method for detecting the change of the end-to-end heterogeneous remote sensing image based on the domain transformation according to claim 1, wherein the constructing of the domain transformation network in the step 2 is specifically as follows:
the domain transformation network is a conditional generation countermeasure network, and has two generators G with the same structure 1 And G 2 Two identical-structure discriminators D 1 And D 2 . The generator network consists of a U-Net structure, a space transducer module and a time transducer module, and the discriminator network consists of a multi-scale discriminator.
4. The method for detecting the change of the end-to-end heterogeneous remote sensing image based on domain transformation according to claim 1, wherein the construction of the change detection network in the step 3 is specifically as follows:
the change detection network has a dual-branch twin structure sharing weights, each branch consisting of one convolution block and three residual convolution blocks. The network also has three transformers modules for capturing long-distance dependencies and global dependencies and one feature fusion module.
5. The method for detecting the change of the end-to-end heterogeneous remote sensing image based on domain transformation according to claim 1, wherein the step 4 is characterized in that the segmented pair sub-images are input into a domain transformation network, so that the two images are in the same image domain, and the method is specifically as follows:
randomly initializing network parameters of two generators and two multi-scale discriminators;
inputting two paired heterologous remote sensing images X and Y into a domain transformation network;
the two transformed images X are obtained through the repeated countermeasure iteration training of the network 1 And Y 1 Wherein X and X 1 Belonging to the same image domain, Y and Y 1 Belonging to another identical image domain.
6. The method for detecting the change of the end-to-end heterogeneous remote sensing image based on domain transformation according to claim 1, wherein in step 5, the image pairs in the same image domain are input into a change detection network to obtain a binary change prediction graph, and the binary change prediction graph is specifically as follows:
randomly initializing network parameters of a change detection network;
selecting a group of image input change detection networks which are easy to process according to the two groups of images in the same domain obtained in the step 4;
three groups of multi-scale features F are obtained through two images of the twin structure 1 ，F 2 ，F 3 Obtaining three multi-scale difference features D through element level difference 1 ，D 2 ，D 3 ；
Three multi-scale difference features D 1 ，D 2 ，D 3 Respectively entering three transducer modules;
the outputs of the three transformers enter a characteristic fusion module for fusion;
and classifying the fused features to obtain a final binary change prediction graph.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311587728.4A CN117788379A (en) | 2023-11-27 | 2023-11-27 | Domain transformation-based end-to-end heterogeneous remote sensing image change detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311587728.4A CN117788379A (en) | 2023-11-27 | 2023-11-27 | Domain transformation-based end-to-end heterogeneous remote sensing image change detection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117788379A true CN117788379A (en) | 2024-03-29 |
Family
ID=90388101
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311587728.4A Pending CN117788379A (en) | 2023-11-27 | 2023-11-27 | Domain transformation-based end-to-end heterogeneous remote sensing image change detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117788379A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118212534A (en) * | 2024-05-17 | 2024-06-18 | 长江水利委员会网络与信息中心 | Method and system for detecting change of double-time-phase remote sensing image |
-
2023
- 2023-11-27 CN CN202311587728.4A patent/CN117788379A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118212534A (en) * | 2024-05-17 | 2024-06-18 | 长江水利委员会网络与信息中心 | Method and system for detecting change of double-time-phase remote sensing image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110533631B (en) | SAR image change detection method based on pyramid pooling twin network | |
| CN110929607B (en) | Remote sensing identification method and system for urban building construction progress | |
| CN109523470B (en) | Depth image super-resolution reconstruction method and system | |
| CN111080629A (en) | Method for detecting image splicing tampering | |
| CN111160171B (en) | Radiation source signal identification method combining two-domain multi-features | |
| CN112861720A (en) | Remote sensing image small sample target detection method based on prototype convolutional neural network | |
| CN102629374B (en) | Image super resolution (SR) reconstruction method based on subspace projection and neighborhood embedding | |
| CN108229551B (en) | Hyperspectral remote sensing image classification method based on compact dictionary sparse representation | |
| Gou et al. | Remote sensing image super-resolution reconstruction based on nonlocal pairwise dictionaries and double regularization | |
| CN117788379A (en) | Domain transformation-based end-to-end heterogeneous remote sensing image change detection method | |
| CN111639587A (en) | Hyperspectral image classification method based on multi-scale spectrum space convolution neural network | |
| Zhao et al. | ADRN: Attention-based deep residual network for hyperspectral image denoising | |
| CN116797787B (en) | Remote sensing image semantic segmentation method based on cross-modal fusion and graph neural network | |
| CN114998703B (en) | Remote sensing image change detection method based on high-resolution convolutional neural network | |
| CN113988147A (en) | Multi-label classification method and device for remote sensing image scene based on graph network, and multi-label retrieval method and device | |
| CN114266957A (en) | Hyperspectral image super-resolution restoration method based on multi-degradation mode data augmentation | |
| CN103226825A (en) | Low-rank sparse model-based remote sensing image change detection method | |
| CN117975133A (en) | Hyperspectral image classification method, hyperspectral image classification system and hyperspectral image classification computer program product | |
| CN114612315A (en) | High-resolution image missing region reconstruction method based on multi-task learning | |
| CN116486273B (en) | Method for extracting water body information of small sample remote sensing image | |
| CN116485802B (en) | Insulator flashover defect detection method, device, equipment and storage medium | |
| CN111275680B (en) | SAR image change detection method based on Gabor convolution network | |
| Mu et al. | A two-branch network combined with robust principal component analysis for hyperspectral image classification | |
| CN116977747A (en) | Small sample hyperspectral classification method based on multipath multi-scale feature twin network | |
| CN115100451B (en) | Data expansion method for monitoring oil leakage of hydraulic pump |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |