CN112926624A - Robust multi-temporal multispectral image change detection method - Google Patents

Abstract

The invention discloses a robust multi-temporal multispectral image change detection method, which comprises the steps of firstly, carrying out a correction algorithm of a non-uniform image, respectively unmixing a dual-temporal image through a spectral image unmixing algorithm, and obtaining sub-pixel abundance and an end member matrix; pre-detecting original data by using a k nearest neighbor method, and dividing the data into changed data and unchanged data; calculating a correction matrix by using the abundance of the undetected unchanged data; then, multispectral image change detection based on space-time spectral feature learning is carried out, each pixel in the multi-temporal image is taken as the center, and an S multiplied by S pixel block is taken; pre-detecting original data by using a k nearest neighbor method, and taking changed and unchanged distributed labels as a pseudo training set; training the proposed neural network by using a pseudo training set; and finally, detecting all data by using the trained neural network. The method provided by the invention can effectively resist the influence caused by the non-uniform images and improve the algorithm robustness.

Description

Robust multi-temporal multispectral image change detection method

Technical Field

The invention belongs to the technical field of machine learning, and particularly relates to an image change detection method.

Background

Remote sensing has become a practical, advanced spatial detection technique since the 20 th century. The multi-temporal remote sensing image can enable people to observe and analyze the change of the earth. The difference is analyzed quantitatively and qualitatively from the remote sensing images of the same area at different times, so that the related information of the ground feature change is obtained, which is the main content of the change detection research. Change detection is not far from us, and it has been applied to aspects of social life, such as city transition monitoring, disaster rescue, agricultural production assessment, military reconnaissance, and the like. Compared with other remote sensing image interpretation technologies, the change detection has more data volume to be processed (multi-temporal image) and stronger data heterogeneity (non-uniform image). In remote sensing images, based on a spectral imaging technology, multispectral and hyperspectral images with the characteristic of 'map-map integration' are favored by researchers. The received optical signals (electromagnetic waves) are decomposed into a plurality of continuous wave bands with certain intervals by the light splitting technology, energy on different wave bands is absorbed by corresponding sensors, so that each wave band is imaged independently, images of all wave bands are naturally arranged according to the size of wavelength, and a multi-dimensional data cube is formed. Each layer of the cube is an image of the same region in a certain wave band, and the radiation value of a certain pixel (i.e. pixel) in different wave bands reflects the spectral information of the ground object at the position.

The development of spectral image change detection has a problem that the image inconsistency is a problem which cannot be avoided. As described above, the spectral image is characterized in that it can provide spectral curves corresponding to different regional features, and the same feature in different time phases may exhibit different spectral characteristics due to the influence of objective factors such as different imaging conditions. This undoubtedly affects the accuracy of the spectral image change detection.

The existing unsupervised change detection method is limited by inconsistent images, so that the existing unsupervised change detection method often cannot show excellent performance. And the supervised algorithm represented by the neural network has certain robustness on the non-uniform images due to the label information. However, since a large amount of manpower and material resources are consumed for making a data set for detecting a change of a spectral image, few data sets are available. Therefore, the key to the problem is how to design an unsupervised spectral image change detection algorithm that can resist image non-uniformity, while not requiring a labeled sample.

Disclosure of Invention

In order to overcome the defects of the prior art, the invention provides a robust multi-temporal multispectral image change detection method, which comprises the steps of firstly carrying out a correction algorithm of a non-uniform image, respectively unmixing a dual-temporal image through a spectral image unmixing algorithm, and obtaining sub-pixel abundance and an end member matrix; pre-detecting original data by using a k nearest neighbor method, and dividing the data into changed data and unchanged data; calculating a correction matrix by using the abundance of the undetected unchanged data; then, multispectral image change detection based on space-time spectral feature learning is carried out, each pixel in the multi-temporal image is taken as the center, and an S multiplied by S pixel block is taken; pre-detecting original data by using a k nearest neighbor method, and taking changed and unchanged distributed labels as a pseudo training set; training the proposed neural network by using a pseudo training set; and finally, detecting all data by using the trained neural network. The method provided by the invention can effectively resist the influence caused by the non-uniform images and improve the algorithm robustness.

The technical scheme adopted by the invention for solving the technical problem comprises the following steps:

step 1: a correction algorithm for the non-uniform image;

step 1-1: given a dual temporal image

And auxiliary time phase image

I, J, wherein, N is the length and width of the image, and N is the number of the wave bands of the image; with X₁As a reference image, for X₂And performing a correction;

step 1-2: pair X by multi-layer non-negative matrix factorization₁And X₂Mixing and separating to obtain X₁And X₂Common end-member matrix of

And X₁、X₂Respective abundance matrix

Wherein P is the number of end members;

step 1-3: to X₁And X₂Difference is made to obtain difference image

Element D in D_i,jIs a difference spectral vector of dimension N; then, all spectral vectors in the difference image are subjected to modulus calculation to obtain a corresponding change amplitude map

Clustering C according to the magnitude of the change amplitude by adopting a k nearest neighbor method, and taking the class with the lowest change amplitude as a high-reliability unchanged sample set G;

step 1-4: solving the correction matrix H ═ diag (H) by using the high-reliability unchanged sample set G₁,h₂…h_p…h_P) P is 1,2, …, P; wherein h is_pCalculating according to the formula (1):

wherein

And

are each X₁And X₂Average of the p-th abundance of unchanged pixels of (a);

step 1-5: using the correction matrix H to solve the corrected image X₂′：

X₂′＝A₂×H×E_s

Step 1-6: repeating steps 1-2 to 1-5, and adding X₂Is replaced by X₃Obtaining a corrected image X₃′；

Step 2: a multispectral image change detection method based on space-time spectral feature learning;

step 2-1: to X₁And X₂Performing subtraction to obtain a difference image, performing modulo calculation on all spectral vectors in the difference image to obtain a corresponding change amplitude map, clustering the change amplitude map according to the change amplitude by adopting a k nearest neighbor method, taking the class with the highest change amplitude as a high-reliability change sample, taking the class with the lowest change amplitude as an unchanged sample, and forming a high-reliability pseudo training sample set by using the high-reliability change sample and the unchanged sample;

step 2-2: to X₁、X₂' and X₃Each pixel in the' takes the pixel block of S multiplied by S in the central neighborhood by taking the pixel as the center, and the area which is less than S multiplied by S is filled up in an image mirror mode; then to X₁、X₂' and X₃' in which the same pixel is at X₁、X₂' and X₃Wherein, each pixel block of S multiplied by S corresponds to the pixel point at the position;

step 2-3: taking the block of pixels of S × S corresponding to each pixel position obtained in step 8 as an input sample, and taking the high-reliability changed samples and unchanged samples in the high-reliability pseudo-training sample set obtained in step 7 as labels to train the neural network, wherein the neural network has a structure shown in table 1:

TABLE 1 neural network architecture

And step 3: and (3) processing the multi-temporal image to be detected according to the correction algorithm of the non-uniform image in the step (1), obtaining an S multiplied by S pixel block corresponding to each pixel position in the step (2-2), inputting the pixel block into the neural network trained in the step (2-3), and outputting to obtain a final detection result.

The invention has the following beneficial effects:

1. and (4) no supervision is carried out. The invention generates the pseudo training set by using a pre-detection algorithm to train the neural network, and compared with a supervision algorithm, the invention can carry out detection without introducing prior information.

2. And is more robust. The correction algorithm provided by the invention can effectively resist the influence caused by non-uniform images and improve the robustness of the algorithm, and the multispectral image change detection method can deeply mine time, space and spectrum related information of the multispectral spectral image, extract the space-time spectral characteristics and further ensure the overall superiority and robustness of the algorithm.

3. And is more flexible. The two sub-methods can be used as a whole to carry out change detection on multi-temporal spectral images and can also be split, wherein the correction algorithm can be used as a preprocessing step and is combined with other change detection algorithms to improve the performance of other algorithms.

Drawings

FIG. 1 is a flow chart of the method of the present invention.

FIG. 2 is a diagram of a neural network architecture for the method of the present invention.

Detailed Description

The invention is further illustrated with reference to the following figures and examples.

The invention aims to provide an unsupervised robust multi-temporal spectral image change detection method aiming at the defects of the existing algorithm and combining the capability of an unsupervised algorithm that a labeled sample is not needed and a neural network extracts robust features.

As shown in fig. 1, a robust multi-temporal multispectral image change detection method includes the following steps:

step 1: a correction algorithm for the non-uniform image;

step 1-1: given a dual temporal image

And auxiliary time phase image

I, J, wherein, N is the length and width of the image, and N is the number of the wave bands of the image; with X₁As a reference image, for X₂And performing a correction;

step 1-2: pair X by multi-layer non-negative matrix factorization₁And X₂Mixing and separating to obtain X₁And X₂Common end-member matrix of

And X₁、X₂Respective abundance matrix

Wherein P is the number of end members;

step 1-3: to X₁And X₂Difference is made to obtain difference image

Element D in D_i,jIs a difference spectral vector of dimension N; then, all spectral vectors in the difference image are subjected to modulus calculation to obtain a corresponding change amplitude map

Clustering C according to the magnitude of the change amplitude by adopting a k nearest neighbor method, and taking the class with the lowest change amplitude as a high-reliability unchanged sample set G;

step 1-4: solving the correction matrix H ═ diag (H) by using the high-reliability unchanged sample set G₁,h₂…h_p…h_P) P is 1,2, …, P; wherein h is_pCalculating according to the formula (1):

wherein

And

are each X₁And X₂Average of the p-th abundance of unchanged pixels of (a);

step 1-5: using the correction matrix H to solve the corrected image X₂′：

X₂′＝A₂×H×E_s

Step (ii) of1-6: repeating steps 1-2 to 1-5, and adding X₂Is replaced by X₃Obtaining a corrected image X₃′；

Step 2: a multispectral image change detection method based on space-time spectral feature learning;

step 2-1: to X₁And X₂Performing subtraction to obtain a difference image, performing modulo calculation on all spectral vectors in the difference image to obtain a corresponding change amplitude map, clustering the change amplitude map according to the change amplitude by adopting a k nearest neighbor method, taking the class with the highest change amplitude as a high-reliability change sample, taking the class with the lowest change amplitude as an unchanged sample, and forming a high-reliability pseudo training sample set by using the high-reliability change sample and the unchanged sample; the label information of the pseudo training set is as follows: the sample with the highest change amplitude is regarded as a change sample, and the sample with the lowest change amplitude is regarded as an unchanged sample; because the input of the change detection task is a multi-temporal image and the output is a binary image, the 'sample' here means a pixel at a certain fixed position;

step 2-2: to X₁、X₂' and X₃Each pixel in the' takes the pixel block of S multiplied by S in the central neighborhood by taking the pixel as the center, and the area which is less than S multiplied by S is filled up in an image mirror mode; then to X₁、X₂' and X₃' in which the same pixel is at X₁、X₂' and X₃Wherein, each pixel block of S multiplied by S corresponds to the pixel point at the position;

step 2-3: training the neural network shown in fig. 2 by using the S × S pixel block corresponding to each pixel position obtained in step 8 as an input sample, and using the high-reliability changed sample and unchanged sample in the high-reliability pseudo training sample set obtained in step 7 as labels, wherein the neural network has a structure shown in table 1; the neural network structure comprises a three-dimensional convolution module and a cyclic convolution network, so that the method can learn how to mine and utilize space and spectrum information of a pixel block, and simultaneously, model the time sequence correlation provided by multiple time phases, thereby extracting robust space-time spectrum characteristics, and further judging whether the central pixel of the input pixel block changes or not according to the time sequence correlation;

and step 3: and (3) processing the multi-temporal image to be detected according to the correction algorithm of the non-uniform image in the step (1), obtaining an S multiplied by S pixel block corresponding to each pixel position in the step (2-2), inputting the pixel block into the neural network trained in the step (2-3), and outputting to obtain a final detection result.

Supplementary description of the network structure of table 1:

a) input Size describes the dimension channel × depth × height × width of the Input data of the current layer.

b) The dimensions of the null spectral features of the output of Conv4 are 32 × 6 × 1 × 1. Three spatial spectrum features are output by the three branches, converted into one-dimensional vectors and input to the two RNNs, and two spatial spectrum features with the same size are output after modeling time sequence correlation.

c) Two spatio-temporal spectral features are directly stitched together and input to FC 1.

The specific embodiment is as follows:

1. conditions of the experiment

The invention is in the central processing unit

I5-65003.2 GHz CPU, memory 16G, WINDOWS 10 operating system, using MATLAB, python3.7 and pytorch1.0 software to simulate.

The multi-temporal dataset used in the experiment included three, Taizhou, Wuhan and Nanchang, where Taizhou data were proposed by Zhang et al in the literature "a Coarse-to-Fine Semi-redundant Change Detection for Multispectral Images, IEEE Transactions on Geoscience and movement Sensing, vol.56, No.6, June 2018", and Wuhan and Nanchang were all made by self-collection. The Nanchang data set includes images of three phases.

2. Content of the experiment

In order to illustrate the effectiveness of the correction algorithm for non-uniform images, the correction algorithm is combined with a simple three-dimensional convolutional neural network to obtain a new model called Re-CD, and a new correction algorithm called Re-CD is obtained by selecting a classic Change vector analysis method (CVA) mentioned in the document "Change vector analysis: An ap detecting for Change with a landsat,1980," and a PCA-CVA proposed in the document "Change vector analysis using PCA and inverse triangular function-based simulation. Defence Science Journal, vol.62, No.4, pp.236-242,2012," and a simple neural network (CNN) as a comparison algorithm, and a comparison set Re-CVA of the CVA is also provided, namely, the comparison algorithm Re-CVA is combined with the new correction algorithm. The experimental evaluation indexes include total accuracy (OA), consistency coefficient (Kappa), Unchanged pixel recognition rate (Unchanged), and Changed pixel recognition rate (Changed). The comparative results are shown in tables 1 and 2.

It can be seen that Re-CD has the best performance in unsupervised algorithms, while the performance on OA terms approaches supervised CNN. In addition, compared with the CVA, the Re-CVA formed by combining the correction algorithm with the CVA is greatly improved. This fully accounts for the effectiveness of the correction algorithm.

TABLE 1 Wuhan data set Experimental results

	OA	Kappa	Unchanged	Changed
					CVA	62.56％	0.0400	62.52％	63.95％
Re-CVA	87.95％	0.2598	88.08％	83.65％
					PCA	91.45％	0.3563	91.54％	88.72％
Re-CD	96.37％	0.5627	96.81％	82.35％
					CNN	97.92％	0.7021	98.33％	84.77％

TABLE 2 Taizou data set experimental results

	OA	Kappa	Unchanged	Changed
					CVA	65.10％	0.0456	73.99％	31.09％
Re-CVA	96.05％	0.8733	99.26％	83.75％
					PCA	96.11％	0.8763	99.03％	84.92％
Re-CD	96.22％	0.8772	99.85％	82.33％
					CNN	97.65％	0.9260	99.70％	89.78％

In order to illustrate the effectiveness of the multispectral image change detection method based on space-time spectral feature learning, experiments are carried out on a Nanchang three-time phase data set. Meanwhile, in a comparison method, a correction algorithm is combined with a multispectral image change detection method based on space-time spectral feature learning, and MT-CD is provided. As can be seen from Table 3, MT-CD has a great improvement over Re-CD, and has the best performance in the unsupervised algorithm. In addition, the difference in OA is only around 0.56% compared to supervised CNNs. This fully demonstrates the effectiveness of the multi-spectral image change detection algorithm based on spatio-temporal spectral feature learning.

TABLE 3 Nanchang data set Experimental results

	OA	Kappa	Unchanged	Changed
					CVA	76.90％	0.5271	73.53％	83.23％
Re-CVA	94.95％	0.8909	93.91％	96.91％
					PCA	90.59％	0.7957	91.03％	89.78％
Re-CD	95.21％	0.8925	98.32％	89.39％
					MT-CD	96.52％	0.9227	98.18％	93.42％
CNN	97.08％	0.9347	99.37％	92.78％

In conclusion, the effectiveness of the invention is fully verified.

Claims (1)

1. A robust multi-temporal multispectral image change detection method is characterized by comprising the following steps:

step 1: a correction algorithm for the non-uniform image;

step 1-1: given two-time phase imagesImage

And auxiliary time phase image

I, J, wherein, N is the length and width of the image, and N is the number of the wave bands of the image; with X₁As a reference image, for X₂And performing a correction;

step 1-2: pair X by multi-layer non-negative matrix factorization₁And X₂Mixing and separating to obtain X₁And X₂Common end-member matrix of

And X₁、X₂Respective abundance matrix

Wherein P is the number of end members;

step 1-3: to X₁And X₂Difference is made to obtain difference image

Element D in D_i,jIs a difference spectral vector of dimension N; then, all spectral vectors in the difference image are subjected to modulus calculation to obtain a corresponding change amplitude map

Clustering C according to the magnitude of the change amplitude by adopting a k nearest neighbor method, and taking the class with the lowest change amplitude as a high-reliability unchanged sample set G;

step 1-4: solving the correction matrix H ═ diag (H) by using the high-reliability unchanged sample set G₁,h₂…h_p…h_P) P is 1,2, …, P; wherein h is_pCalculating according to the formula (1):

wherein

And

are each X₁And X₂Average of the p-th abundance of unchanged pixels of (a);

step 1-5: using the correction matrix H to solve the corrected image X₂′：

X₂′＝A₂×H×E_s

Step 1-6: repeating steps 1-2 to 1-5, and adding X₂Is replaced by X₃Obtaining a corrected image X₃′；

Step 2: a multispectral image change detection method based on space-time spectral feature learning;

step 2-1: to X₁And X₂Performing subtraction to obtain a difference image, performing modulo calculation on all spectral vectors in the difference image to obtain a corresponding change amplitude map, clustering the change amplitude map according to the change amplitude by adopting a k nearest neighbor method, taking the class with the highest change amplitude as a high-reliability change sample, taking the class with the lowest change amplitude as an unchanged sample, and forming a high-reliability pseudo training sample set by using the high-reliability change sample and the unchanged sample;

step 2-2: to X₁、X₂' and X₃Each pixel in the' takes the pixel block of S multiplied by S in the central neighborhood by taking the pixel as the center, and the area which is less than S multiplied by S is filled up in an image mirror mode; then to X₁、X₂' and X₃' in which the same pixel is at X₁、X₂' and X₃Wherein, each pixel block of S multiplied by S corresponds to the pixel point at the position;

step 2-3: taking the block of pixels of S × S corresponding to each pixel position obtained in step 8 as an input sample, and taking the high-reliability changed samples and unchanged samples in the high-reliability pseudo-training sample set obtained in step 7 as labels to train the neural network, wherein the neural network has a structure shown in table 1:

TABLE 1 neural network architecture

And step 3: and (3) processing the multi-temporal image to be detected according to the correction algorithm of the non-uniform image in the step (1), obtaining an S multiplied by S pixel block corresponding to each pixel position in the step (2-2), inputting the pixel block into the neural network trained in the step (2-3), and outputting to obtain a final detection result.

Images